Chapter 2 : Backend Enterprise Ready Development

Enterprise Requirements for APIs and Microservices

Code Coverage - Jacoco

Generating Code Coverage Locally with JaCoCo, which lateron can be published on SonarQube.

Jacoco Configuration and Features

Build emapi project with Project Configuration,

Code Coverage - JaCoCo : Include [✅] Enable [✅]

Faimliarize with Jacoco related configurations in project:

View emapi\pom.xml, it will have
-  section
- module added: <module>coverage</module>
- plugin jacoco-maven-plugin configured

Jacoco Settings

Make sure properties are set,

<skipUTs>false</skipUTs>
<skipITs>false</skipITs>
<jacoco.skip>false</jacoco.skip>

Jacoco exclusions

If you want to add exclusions for calculating code coverage,

Uncomment/Use property like below, (define on 1 line)
<sonar.coverage.exclusions>**/*ServiceExtend*.*,**/*ServiceCloud*.*,**/*Predicate*.*,**/*Notify*.*,**/*SpringApp*.*</sonar.coverage.exclusions>
And Uncomment in plugin jacoco-maven-plugin

                <configuration>
                    <excludes>${sonar.coverage.exclusions}</excludes> 
                </configuration>

Run Jacoco & View Coverage

To Run Jacoco : Build project - verify
- mvn clean verify
- Generates code coverage
Run Jacoco
- Coverage Module present

Report Location - Aggregate Report

Coverage Report Aggregate
- emapi\coverage\target\site\jacoco-aggregate\index.html

Coverage Report : dbrest
- emapi\app\dbrest\target\site\jacoco\index.html

Coverage Report : dbgraphql
- emapi\app\dbgraphql\target\site\jacoco\index.html

Jacoco Next Steps

Take-home Exercises
- Increase code coverage to 70-80%
  - Add more UT, IT
- Migrate Integration Tests from JUnit to TestNG

SonarScanner - SonarQube

Run Sonar Scanner with SonarQube Server in docker

Install/Setup and Run SonarQube Server in docker

docker run --name sonarqube-custom -p 9000:9000 sonarqube:community

Login to http://localhost:9000/
- login: admin, password: admin
- 1st login will ask to change password. Change to admin11

Projects Screen

Add project on SonarQube server
- Select Manually
- Create project emapi

Click Setup Project, Continue

Analyze your project Screen
- Provide token, Generate a token, Continue
- Select Run analysis on your project, Select Maven

Project Created Screen

Copy the maven command shown (bring it on 1 line for use on Windows)

e.g. command

mvn sonar:sonar -Dsonar.projectKey=emapi -Dsonar.host.url=http://localhost:9000 -Dsonar.login=240ba1c72e4e403614fe74567210c28076380a8a

Customize the command for emapi project as below

mvn org.sonarsource.scanner.maven:sonar-maven-plugin:5.0.0.4389:sonar -DskipUTs=false -DskipITs=false -Djacoco.skip=false -Dsonar.skip=false -Dsonar.projectKey=emapi -Dsonar.host.url=http://localhost:9000 -Dsonar.login=240ba1c72e4e403614fe74567210c28076380a8a -Dsonar.exclusions=**/*Test*/**

Run Sonar Scanner Maven

mvn clean verify -DskipUTs=false -DskipITs=false -Djacoco.skip=false jacoco:report jacoco:report-aggregate

Run the maven goal sonar:sonar customized command given above.
- Repeat run above command if getting error.

Sample output of mvn `sonar:sonar` e.g.

[INFO] --- sonar:5.0.0.4389:sonar (default-cli) @ emapi ---
[INFO] Java 17.0.10 Microsoft (64-bit)
[INFO] Windows 11 10.0 (amd64)
[INFO] User cache: C:\Users\Computer01\.sonar\cache
[INFO] Communicating with SonarQube Server 8.9.10.61524
[INFO] Load global settings
[INFO] Load global settings (done) | time=65ms
[INFO] Server id: BF41A1F2-AZTqdihJHPaIwNla_NrV
[INFO] User cache: C:\Users\Computer01\.sonar\cache
[INFO] Load/download plugins
[INFO] Load plugins index
[INFO] Load plugins index (done) | time=32ms
[INFO] Load/download plugins (done) | time=78ms
[INFO] Process project properties
[INFO] Process project properties (done) | time=6ms
[INFO] Execute project builders
[INFO] Execute project builders (done) | time=10ms
[INFO] Project key: emapi
[INFO] Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi
[INFO] Working dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\target\sonar
[INFO] Load project settings for component key: 'emapi'
[INFO] Load project settings for component key: 'emapi' (done) | time=22ms
[INFO] Load quality profiles
[INFO] Load quality profiles (done) | time=35ms
[INFO] Load active rules
[INFO] Load active rules (done) | time=1066ms
[WARNING] SCM provider autodetection failed. Please use "sonar.scm.provider" to define SCM of your project, or disable the SCM Sensor in the project settings.
[INFO] Indexing files...
[INFO] Project configuration:
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : app : dbgraphql'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\app\dbgraphql
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : app : dbrest'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\app\dbrest
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : app'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\app
[INFO]   Source paths: pom.xml
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : lib : base-app'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\lib\base-app
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : lib'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\lib
[INFO]   Source paths: pom.xml
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : testApi : dbgraphqlTest'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\testApi\dbgraphqlTest
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: src/test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : testApi : dbrestTest'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\testApi\dbrestTest
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: src/test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : testApi'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\testApi
[INFO]   Source paths: pom.xml
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi
[INFO]   Source paths: pom.xml
[INFO]   Excluded sources: **/*Test*/**
[INFO] 140 files indexed
[INFO] 14 files ignored because of inclusion/exclusion patterns
[INFO] Quality profile for java: Sonar way
[INFO] Quality profile for xml: Sonar way
[INFO] ------------- Run sensors on module emapi : lib : base-app
[INFO] Load metrics repository
[INFO] Load metrics repository (done) | time=29ms
[INFO] Sensor JavaSquidSensor [java]
[INFO] Configured Java source version (sonar.java.source): 17
[INFO] JavaClasspath initialization
[INFO] JavaClasspath initialization (done) | time=10ms
[INFO] JavaTestClasspath initialization
[INFO] JavaTestClasspath initialization (done) | time=0ms
[INFO] Java Main Files AST scan
[INFO] 50 source files to be analyzed
[INFO] Load project repositories
[INFO] Load project repositories (done) | time=36ms
[INFO] 50/50 source files have been analyzed
[INFO] Java Main Files AST scan (done) | time=7050ms
[INFO] Java Test Files AST scan
[INFO] 6 source files to be analyzed
[INFO] 6/6 source files have been analyzed
[INFO] Java Test Files AST scan (done) | time=424ms
[INFO] Java Generated Files AST scan
[INFO] 0 source files to be analyzed
[INFO] 0/0 source files have been analyzed
[INFO] Java Generated Files AST scan (done) | time=0ms
[INFO] Sensor JavaSquidSensor [java] (done) | time=7694ms
[INFO] Sensor CSS Rules [cssfamily]
[INFO] No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
[INFO] Sensor CSS Rules [cssfamily] (done) | time=0ms
[INFO] Sensor JaCoCo XML Report Importer [jacoco]
[INFO] 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
[INFO] Importing 1 report(s). Turn your logs in debug mode in order to see the exhaustive list.
[INFO] Sensor JaCoCo XML Report Importer [jacoco] (done) | time=157ms
[INFO] Sensor C# Project Type Information [csharp]
[INFO] Sensor C# Project Type Information [csharp] (done) | time=0ms
[INFO] Sensor C# Properties [csharp]
[INFO] Sensor C# Properties [csharp] (done) | time=0ms
[INFO] Sensor SurefireSensor [java]
[INFO] parsing [C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\lib\base-app\target\surefire-reports]
[INFO] Sensor SurefireSensor [java] (done) | time=126ms
[INFO] Sensor JavaXmlSensor [java]
[INFO] 1 source file to be analyzed
[INFO] 1/1 source file has been analyzed
[INFO] Sensor JavaXmlSensor [java] (done) | time=110ms
[INFO] Sensor HTML [web]
[INFO] Sensor HTML [web] (done) | time=0ms
[INFO] Sensor XML Sensor [xml]
[INFO] 1 source file to be analyzed
[INFO] 1/1 source file has been analyzed
[INFO] Sensor XML Sensor [xml] (done) | time=110ms
[INFO] Sensor VB.NET Project Type Information [vbnet]
[INFO] Sensor VB.NET Project Type Information [vbnet] (done) | time=0ms
[INFO] Sensor VB.NET Properties [vbnet]
[INFO] Sensor VB.NET Properties [vbnet] (done) | time=0ms
...
[INFO] ------------- Run sensors on project
[INFO] Sensor Zero Coverage Sensor
[INFO] Sensor Zero Coverage Sensor (done) | time=0ms
[INFO] Sensor Java CPD Block Indexer
[INFO] Sensor Java CPD Block Indexer (done) | time=126ms
[INFO] SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.
[INFO] CPD Executor 11 files had no CPD blocks
[INFO] CPD Executor Calculating CPD for 56 files
[INFO] CPD Executor CPD calculation finished (done) | time=47ms
[INFO] Analysis report generated in 204ms, dir size=1 MB
[INFO] Analysis report compressed in 3944ms, zip size=458 KB
[INFO] Analysis report uploaded in 88ms
[INFO] ANALYSIS SUCCESSFUL, you can browse http://localhost:9000/dashboard?id=emapi
[INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
[INFO] More about the report processing at http://localhost:9000/api/ce/task?id=AZTquy79HPaIwNla_ScX
[INFO] Analysis total time: 19.891 s
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for emapi 1.0-SNAPSHOT:
[INFO]
[INFO] emapi .............................................. SUCCESS [ 22.294 s]
[INFO] emapi : lib ........................................ SKIPPED
[INFO] emapi : lib : base-app ............................. SKIPPED
[INFO] emapi : app ........................................ SKIPPED
[INFO] emapi : app : dbrest ............................... SKIPPED
[INFO] emapi : app : dbgraphql ............................ SKIPPED
[INFO] emapi : testApi .................................... SKIPPED
[INFO] emapi : testApi : dbrestTest ....................... SKIPPED
[INFO] emapi : testApi : dbgraphqlTest .................... SKIPPED
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
------------------------------------------------------------------------

View SonarQube Analysis Report

e.g. at http://localhost:9000/dashboard?id=emapi

View SonarQube Analysis Code Tab

e.g. at http://localhost:9000/code?id=emapi

Maven Dependency Analysis

Run Maven Dependency Analysis

Refer to generated Jenkinsfile in Spring Java

backend\spring-java\emapi\Jenkinsfile

Refer to stage CI Run Apache Maven Dependency Analysis

Run Maven Dependency Analysis

mvn org.apache.maven.plugins:maven-dependency-plugin:analyze-report

...
[INFO] <<< dependency:3.6.1:analyze-report (default-cli) < test-compile @ dbrest <<<
...
[INFO] --- dependency:3.6.1:analyze-report (default-cli) @ dbrest ---
...

Report Location
- Report at target/site/dependency-analysis.html
- e.g. spring-java\emapi\app\dbrest\target\site\dependency-analysis.html

tests API Tests - TestNG, WebClient

TestNG is a powerful testing framework, you can use it for acceptance or regression testing.

Refer to web site and docs : TestNG | TestNG Docs

Configuring and Using TestNG

Java-Spring Release includes for emapi project, API Tests - using TestNG, WebClient.

Build emapi project with Project Configuration,

tests API Tests - TestNG, WebClient : Include [✅] Enable []

NOTE: The API tests need the APIs project built and running live against database connection, unlike unit tests. So you will need to enable property as below once APIs are running, and then run TestNG tests.

`emapi` project prepare for TestNG

Steps: Build Project

cd emapi
mvn package

Configurations in project:

View/Edit emapi\pom.xml, Make sure properties are set,

        <!-- skip testNG API tests maven failsafe property for testApi -->
        <skipATs>false</skipATs>

Run APIs Live For testApi pre-requisite

Steps:

1) In new CMD prompt, Run dbrest APIs

java -jar app\dbrest\target\dbrest-1.0-SNAPSHOT.jar
...
2025-04-02T11:21:31.644+05:30  INFO 79236 --- [           main] c.e.emapi.app.EmDbRestAppRestSpringApp   : Started EmDbRestAppRestSpringApp ...

2) In new CMD prompt, Run dbgraphql APIs

java -jar app\dbgraphql\target\dbgraphql-1.0-SNAPSHOT.jar
...
2025-04-02T11:30:49.458+05:30  INFO 77216 --- [           main] c.e.emapi.app.EmDbGraphQLAppSpringApp    : Started EmDbGraphQLAppSpringApp ...

Run testApi testNG tests

1) In new CMD prompt or via IDE, run maven command

cd emapi
mvn verify

dbrestTest for `dbrest`

dbrestTest - Report Location

Location Path

emapi\testApi\dbrestTest\target\failsafe-reports\index.html

dbrestTest - Report Location

dbrestTest - Report

dbrestTest - Report

dbrestTest - Emailable Report

Location Path

testApi\dbrestTest\target\failsafe-reports\emailable-report.html

dbrestTest - Emailable Report

dbgraphqlTest - Report

dbgraphqlTest - Report

dbgraphqlTest - Emailable Report

dbgraphqlTest - Emailable Report

Failsafe Aggregate Report

To include testApi tests run, in overall Failsafe Aggregate Report, which will include IT Integration Tests as well.

Steps: In new CMD prompt, Run command

cd emapi
mvn surefire-report:failsafe-report-only -Daggregate=true

Location of Failsafe Aggregate Report:

emapi\target\site\failsafe-report.html

Failsafe Aggregate Test report

Failsafe Aggregate Test report

API Logging

API Logging is included and available in emapi project.

e.g. API endpoint call - Query

curl -X 'GET' \
  'http://127.0.0.1:9080/emdbrest/tenant/Query?tenantId=3' \
  -H 'accept: application/hal+json'

Check API Logging

2024-10-02T16:57:21.469+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.w.f.CommonsRequestLoggingFilter      : Before request [GET /emdbrest/tenant/Query?tenantId=3]
2024-10-02T16:57:21.470+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.web.servlet.DispatcherServlet        : GET "/emdbrest/tenant/Query?tenantId=3", parameters={masked}
2024-10-02T16:57:21.471+05:30 DEBUG 35656 --- [io-9080-exec-10] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.example.emapi.app.Tenant.TenantDataRestController#TenantQuery(long)
2024-10-02T16:57:22.067+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Using 'application/hal+json', given [application/hal+json] and supported [application/json, application/*+json]
2024-10-02T16:57:22.067+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Writing [[Tenant [ tenantId = 3 ]]]
2024-10-02T16:57:22.071+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.web.servlet.DispatcherServlet        : Completed 200 OK
2024-10-02T16:57:22.072+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.w.f.CommonsRequestLoggingFilter      : API LOGGING: GET /emdbrest/tenant/Query?tenantId=3]

e.g. API endpoint call - Create with Payload

curl -X 'POST' \
  'http://127.0.0.1:9080/emdbrest/tenant/Create' \
  -H 'accept: application/hal+json' \
  -H 'Content-Type: application/json' \
  -d '  {
    "tenantId": 13,
    "customerId": 2,
    "tenantName": "Manipal - Pune Branch 2",
    "beginDate": "2024-09-19",
    "minOrderQuantity": 10,
    "customizationRequired": 1,
    "custMicrosvcType": "synchronous-webclient",
    "customizationService": "tenant-2-validation-service",
    "customizationPayload": "order_data"
  }'

Check API Logging with Payload

2024-10-02T17:06:11.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.w.f.CommonsRequestLoggingFilter      : Before request [POST /emdbrest/tenant/Create]
2024-10-02T17:06:11.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.web.servlet.DispatcherServlet        : POST "/emdbrest/tenant/Create", parameters={}
2024-10-02T17:06:11.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.example.emapi.app.Tenant.TenantDataRestController#TenantCreate(Tenant)
2024-10-02T17:06:11.249+05:30 DEBUG 35656 --- [nio-9080-exec-7] m.m.a.RequestResponseBodyMethodProcessor : Read "application/json;charset=UTF-8" to [Tenant [ tenantId = 13 ]]
2024-10-02T17:06:11.488+05:30  INFO 35656 --- [nio-9080-exec-7] c.e.e.a.Tenant.TenantDataRestController  : Tenant Begin Create Record For [Tenant [ tenantId = 13 ]] 
2024-10-02T17:06:11.746+05:30  INFO 35656 --- [nio-9080-exec-7] c.e.emapi.app.Tenant.TenantService       : Tenant Created Record [Tenant [ tenantId = 13 ]] 
2024-10-02T17:06:12.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Using 'application/hal+json', given [application/hal+json] and supported [application/json, application/*+json]
2024-10-02T17:06:12.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Writing [Tenant [ tenantId = 13 ]]
2024-10-02T17:06:12.230+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.web.servlet.DispatcherServlet        : Completed 201 CREATED
2024-10-02T17:06:12.231+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.w.f.CommonsRequestLoggingFilter      : API LOGGING: POST /emdbrest/tenant/Create, payload=  {
    "tenantId": 13,
    "customerId": 2,
    "tenantName": "Manipal - Pune Branch 2",
    "beginDate": "2024-09-19",
    "minOrderQuantity": 10,
    "customizationRequired": 1,
    "custMicrosvcType": "synchronous-webclient",
    "customizationService": "tenant-2-validation-service",
    "customizationPayload": "order_data"
  }]

Note:

API Payload Logging is set for max length 10000. Adjust setting or disable it in file:
- emapi\lib\base-app\src\main\java\com\example\emapi\app\EmApiRequestLoggingFilterConfig.java

Security - OAuth2 with Keycloak

caution

Keycloak server needs to be accessible and running.
Run Keycloak in Docker
Using template for it, More info
- Tutorial 3 : EasyManage Templates

Run Keycloak in Docker

Run Keycloak in Docker

Build emapi project with Project Configuration,

Security Configuration :
- OAuth2 [Authorization Server]
Compile and Run project
Verify APIs security
Open Swagger API explorer and test API endpoints
- http://127.0.0.1:9080/swagger-ui/index.html
- Note: Swagger UI endpoints displaying is permitted without security.

Test APIs without auth token

curl -X 'GET' \
  'http://127.0.0.1:9080/emdbrest/erp_product/ViewAll' \
  -H 'accept: application/hal+json'

Server response:

Code    Details
401   Error: response status is 401

Log shows:

2024-10-02T17:50:42.428+05:30 DEBUG 36388 --- [nio-9080-exec-6] o.s.security.web.FilterChainProxy        : Securing GET /emdbrest/erp_product/ViewAll
2024-10-02T17:50:42.428+05:30 DEBUG 36388 --- [nio-9080-exec-6] o.s.s.w.a.AnonymousAuthenticationFilter  : Set SecurityContextHolder to anonymous SecurityContext
2024-10-02T17:50:42.428+05:30 DEBUG 36388 --- [nio-9080-exec-6] o.s.s.w.a.i.FilterSecurityInterceptor    : Failed to authorize filter invocation [GET /emdbrest/erp_product/ViewAll] with attributes [hasRole('ROLE_USER')]
2024-10-02T17:50:42.429+05:30 DEBUG 36388 --- [nio-9080-exec-6] o.s.s.w.s.HttpSessionRequestCache        : Saved request http://127.0.0.1:9080/emdbrest/erp_product/ViewAll?continue to session

Get auth token: Giving client_secret and password values

Note: Token expires in 60 sec by default

curl -d "client_id=login-app&client_secret=<client_secret>&username=emUser&password=<password>&grant_type=password" http://127.0.0.1:8180/realms/emapi/protocol/openid-connect/token

Representative Output: Trimmed for displaying here ...

{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJNaU5KUmhtODZiYlZkUk82bGdiVC1EM3c5WEM4U2pkdnJBRndZYkVBaWFJIn0
...
rTmF-Ld1LMiksm8QBBsYGaZOKcWIiXXRuGhGD4RahbhnU6ckQoGD1LPnLNUHLwrTJT7YUW_1dCDMi95231q_MP4XFpmzCGpxBx4tRFmPf2Ag","expires_in":300,"refresh_expires_in":1800,"refresh_token":"eyJhbGciOiJIUzUxMiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIwN2UwZDk0O
...
kmU-2OxA","token_type":"Bearer","not-before-policy":0,"session_state":"81f96ffc-6a5f-484c-878e-d4456b3af966","scope":"email profile"}

Note auth token to use in curl below.

Test APIs with auth token

curl -H "Accept: application/json" -H "Authorization: Bearer <token>" "http://127.0.0.1:9080/emdbrest/erp_product/Query?productId=1"

Server response:

[{"productId":1,"productName":"string new","productCategory":"string new","primarySupplier":"string","productDesc":"string","productPicture":null}]

Log shows:

2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.security.web.FilterChainProxy        : Securing GET /emdbrest/erp_product/Query?productId=1
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.s.o.s.r.a.JwtAuthenticationProvider  : Authenticated token
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] .s.r.w.a.BearerTokenAuthenticationFilter : Set SecurityContextHolder to JwtAuthenticationToken [Principal=org.springframework.security.oauth2.jwt.Jwt@82025aa3, Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=127.0.0.1, SessionId=null], Granted Authorities=[ROLE_USER]]
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.s.w.a.i.FilterSecurityInterceptor    : Authorized filter invocation [GET /emdbrest/erp_product/Query?productId=1] with attributes [hasRole('ROLE_USER')]
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.security.web.FilterChainProxy        : Secured GET /emdbrest/erp_product/Query?productId=1
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.w.f.CommonsRequestLoggingFilter      : Before request [GET /emdbrest/erp_product/Query?productId=1]
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.web.servlet.DispatcherServlet        : GET "/emdbrest/erp_product/Query?productId=1", parameters={masked}
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.example.emapi.app.ErpProduct.ErpProductDataRestController#ErpProductQuery(long)
2024-10-02T18:11:31.507+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Using 'application/json', given [application/json] and supported [application/json, application/*+json]
2024-10-02T18:11:31.507+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Writing [[ErpProduct [ productId = 1 ]]]
2024-10-02T18:11:31.507+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.web.servlet.DispatcherServlet        : Completed 200 OK
2024-10-02T18:11:31.507+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.w.f.CommonsRequestLoggingFilter      : API LOGGING: GET /emdbrest/erp_product/Query?productId=1]

If token expired, The logs show

2024-10-02T18:08:11.335+05:30 DEBUG 36388 --- [nio-9080-exec-3] o.s.s.oauth2.jwt.JwtTimestampValidator   : Jwt expired at 2024-10-02T12:34:29Z
2024-10-02T18:08:11.335+05:30 DEBUG 36388 --- [nio-9080-exec-3] o.s.s.o.s.r.a.JwtAuthenticationProvider  : Failed to authenticate since the JWT was invalid

Security - OAuth2 with Keycloak With OAuth2 Client

Take-home Exercises
- Implement Security Requirements for below.
- OAuth2 [Authorization Server + OAuth2 Client For Web Apps]
- OAuth2 [Authorization Server + OAuth2 Client For BFF Auth APIs]

Redis cache

caution

Redis server needs to be accessible and running.
Run Redis in Docker
Using template for it, More info
- Tutorial 3 : EasyManage Templates

Run Redis in Docker

Run Redis in Docker

Build emapi project with Project Configuration,

Redis Cache : Include [✅] Enable [✅]

Enable Swagger UI to display Request Duration Swagger UI has the display Request Duration parameter to show how long a request takes. Enable it.
- Edit : emapi\app\dbrest\src\main\resources\application.properties
- Add line springdoc.swagger-ui.display-request-duration=true
Compile and Run project
Verify APIs response times
Open Swagger API explorer and test API endpoints
- http://127.0.0.1:9080/swagger-ui/index.html

Test APIs : 1st call

http://127.0.0.1:9080/emdbrest/erp_customer/Query?customerId=5

Server response in:
Request Duration 
1045 ms

Test APIs : 1st call

Test APIs : 1st call

Test APIs : 2nd call - Which will use redis cached data!

http://127.0.0.1:9080/emdbrest/erp_customer/Query?customerId=5

Server response in:
Request Duration 
104 ms

Test APIs : 2nd call

Test APIs : 2nd call

Serverless

Covered in tutorial : Serverless

Logging - ELK stack

caution

ELK Stack consists of Elasticsearch, Logstash, and Kibana
ELK servers needs to be accessible and running.
Run ELK in Docker
Using template for it, More info
- Tutorial 3 : EasyManage Templates

Run ELK in Docker

Run ELK in Docker

Build emapi project with Project Configuration,

Logging - ELK : Include [✅] Enable [✅]

Add custom logs for observability and monitoring, e.g.:

    logger.info(String.format("ErpCustomer list size=[%s] firstCustomerId=[%s] firstCustomerName=[%s]", ErpCustomerList.size(), ErpCustomerList.get(0).getCustomerId(), ErpCustomerList.get(0).getName()));

Compile and Run project
Verify APIs
Open Swagger API explorer and test API endpoints
- http://127.0.0.1:9080/swagger-ui/index.html

Log shows:

logdate=(2024-10-02T19:25:05,046) thread=(main)) level=(DEBUG) loggerclass=(org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver) message=(ControllerAdvice beans: 2 @ExceptionHandler, 1 ResponseBodyAdvice)
logdate=(2024-10-02T19:25:05,091) thread=(main)) level=(DEBUG) loggerclass=(org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver) message=(ControllerAdvice beans: 2 @ExceptionHandler, 0 ResponseBodyAdvice)
logdate=(2024-10-02T19:25:05,593) thread=(main)) level=(INFO) loggerclass=(com.example.emapi.app.EmDbRestAppRestSpringApp) message=(Started EmDbRestAppRestSpringApp in 9.93 seconds (process running for 11.36))

Test APIs : Make Create calls, with "productId" = 800, 801, 802

curl -X 'POST' \
  'http://127.0.0.1:9080/emdbrest/erp_product/Create' \
  -H 'accept: application/hal+json' \
  -H 'Content-Type: application/json' \
  -d '{
  "productId": 802,
  "productName": "Product 802",
  "productCategory": "Vitamin",
  "primarySupplier": "Cipla"
}'

View Logs in Kibana

Access from Kibana UI: http://localhost:5601

Configure index pattern: logstash-*
Visualize : Create Visualization
- View

DevOps

DevOps CI/CD pipelines for build, verify, deploy - using Jenkins.

CI/CD - Continuous Integration/Continuous Delivery

Jenkins

Refer to generated Jenkinsfile in Spring Java project.

backend\spring-java\emapi\Jenkinsfile

GitLab CI/CD

Refer to generated .gitlab-ci.yml in Spring Java project.

backend\spring-java\emapi\.gitlab-ci.yml

DevSecOps

DevSecOps is an extension of DevOps that integrates security into the entire software development process, specifically into CI/CD pipeline.

DevSecOps Tools

For Backend - Java Spring Boot

Code Coverage - Jacoco

Covered earlier

SonarScanner - SonarQube

SonarScanner used for assessing code quality, analyze the code including

Static Code Analysis
Security Analysis

Covered earlier
Please refer to SonarScanner - SonarQube

Maven Dependency Analysis

Please refer to Maven Dependency Analysis

OWASP dependency check Maven

Note: 1st Run >20Min

Please see Jenkin pipeline OWASP dependency check stage in emapi\Jenkinsfile

Sonatype Lifecycle

Please refer to: EasyManage: Backend Templates
- Locate in generated code folder ...\resources\templates\backend\spring-java
- Readme DevSecOps: backend\spring-java\DevSecOps\README-DevSecOps.md
- Readme Sonatype Lifecycle: backend\spring-java\DevSecOps\sonatype\README-sonatype.md

Please see next section for a brief overview on Sonatype Nexus Scan.

• Sonatype Nexus Scan

Run Sonatype Nexus Scan with Nexus IQ Server in docker

Install/Setup and Run Nexus IQ Server in docker

docker run -d -p 8070:8070 -p 8071:8071 --name nexus-iq-server sonatype/nexus-iq-server

Login to http://localhost:8070/
- login: admin / admin123
- Once running, the IQ Server product license must be installed.

Run Sonatype Nexus Scan

Run the maven goal evaluate

mvn package com.sonatype.clm:clm-maven-plugin:evaluate -Dclm.additionalScopes=test,provided -Dclm.applicationId=emapi -Dclm.serverUrl=http://localhost:8070 -Dclm.username=admin -Dclm.password=admin123

Sample output of mvn `evaluate` e.g.

[INFO] --- clm-maven-plugin::evaluate (default-cli) @ line-comm-03 ---
[INFO] Starting scan...
...
[INFO] Evaluating policies on http://localhost:8070 ...
[ERROR] Sonatype IQ reports policy 'Security-Critical' failing for 
  component 'org.apache.logging.log4j:log4j-core:2.9.0' with hash '052f6548ae1688e126c2' due to 
    constraint 'Critical risk CVSS score': 
      Security Vulnerability Severity >= 9 because: Found security vulnerability CVE-2021-44228 with severity >= 9 (severity = 10.0)
[INFO] ------------------------------------------------------------------------
[INFO] Policy Action: Failure
[INFO] Number of components affected: 1 critical, 0 severe, 0 moderate
[INFO] Number of open policy violations: 2 critical, 0 severe, 1 moderate
[INFO] Number of grandfathered policy violations: 0
[INFO] Number of components evaluated: 5
[INFO] The detailed report can be viewed at: http://localhost:8070/ui/links/application/local-iq-app/report/755d19e970fd491ca2e23f21bea35d58
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
------------------------------------------------------------------------

View Result Of Sonatype Nexus Scan
- Please see details in output log:
  - Policy Action: None | Warn | Failure
  - The detailed report can be viewed at: Given Link

Chapter 2 : Backend Enterprise Ready Development

Enterprise Requirements for APIs and Microservices​

Code Coverage - Jacoco​

Jacoco Configuration and Features​

Jacoco Settings​

Jacoco exclusions​

Run Jacoco & View Coverage​

Jacoco Next Steps​

SonarScanner - SonarQube​

Install/Setup and Run SonarQube Server in docker​

Run Sonar Scanner Maven​

Maven Dependency Analysis​

tests API Tests - TestNG, WebClient​

Configuring and Using TestNG​

emapi project prepare for TestNG​

Run APIs Live For testApi pre-requisite​

Run testApi testNG tests​

dbrestTest for dbrest​

Failsafe Aggregate Report​

API Logging​

Security - OAuth2 with Keycloak​

Security - OAuth2 with Keycloak With OAuth2 Client​

Redis cache​

Serverless​

Logging - ELK stack​

View Logs in Kibana​

DevOps​

CI/CD - Continuous Integration/Continuous Delivery​

Jenkins​

GitLab CI/CD​

DevSecOps​

DevSecOps Tools​

For Backend - Java Spring Boot​

Code Coverage - Jacoco​

SonarScanner - SonarQube​

Maven Dependency Analysis​

OWASP dependency check Maven​

Sonatype Lifecycle​

• Sonatype Nexus Scan​

Install/Setup and Run Nexus IQ Server in docker​

Run Sonatype Nexus Scan​

Enterprise Requirements for APIs and Microservices

Code Coverage - Jacoco

Jacoco Configuration and Features

Jacoco Settings

Jacoco exclusions

Run Jacoco & View Coverage

Jacoco Next Steps

SonarScanner - SonarQube

Install/Setup and Run SonarQube Server in docker

Run Sonar Scanner Maven

Maven Dependency Analysis

tests API Tests - TestNG, WebClient

Configuring and Using TestNG

`emapi` project prepare for TestNG

Run APIs Live For testApi pre-requisite

Run testApi testNG tests

dbrestTest for `dbrest`

Failsafe Aggregate Report

API Logging

Security - OAuth2 with Keycloak

Security - OAuth2 with Keycloak With OAuth2 Client

Redis cache

Serverless

Logging - ELK stack

View Logs in Kibana

DevOps

CI/CD - Continuous Integration/Continuous Delivery

Jenkins

GitLab CI/CD

DevSecOps

DevSecOps Tools

For Backend - Java Spring Boot

Code Coverage - Jacoco

SonarScanner - SonarQube

Maven Dependency Analysis

OWASP dependency check Maven

Sonatype Lifecycle

• Sonatype Nexus Scan

Install/Setup and Run Nexus IQ Server in docker

Run Sonatype Nexus Scan