Chapter 2 : Backend Enterprise Ready Development

Enterprise Requirements for APIs and Microservices

Code Coverage - SonarQube

Generating Code Coverage Locally with JaCoCo, Sonar Scanner

• Run Jacoco (Coverage Module)

• Report Location

• Coverage Report

• Coverage Report : dbrest

• Coverage Report : dbgraphql

• Take-home Exercises
  • Increase code coverage to 70-80%
    • Add more UT, IT
  • Migrate Integration Tests from JUnit to TestNG

SonarScanner - SonarQube

Run Sonar Scanner with SonarQube Server in docker

Install/Setup and Run SonarQube Server in docker

docker run --name sonarqube-custom -p 9000:9000 sonarqube:community

• Login to http://localhost:9000/
  • login: admin, password: admin
  • 1st login will ask to change password. Change to admin11

• Projects Screen

• Add project on SonarQube server
  • Select Manually
  • Create project emapi

• Click Setup Project, Continue

• Analyze your project Screen
  • Provide token, Generate a token, Continue
  • Select Run analysis on your project, Select Maven

• Project Created Screen

• Copy the maven command shown (bring it on 1 line for use on Windows)

e.g. command

mvn sonar:sonar -Dsonar.projectKey=emapi -Dsonar.host.url=http://localhost:9000 -Dsonar.login=240ba1c72e4e403614fe74567210c28076380a8a

Customize the command for emapi project as below

mvn org.sonarsource.scanner.maven:sonar-maven-plugin:5.0.0.4389:sonar -DskipUTs=false -DskipITs=false -Djacoco.skip=false -Dsonar.skip=false -Dsonar.projectKey=emapi -Dsonar.host.url=http://localhost:9000 -Dsonar.login=240ba1c72e4e403614fe74567210c28076380a8a -Dsonar.exclusions=**/*Test*/** 

Prepare emapi project for SonarScanner

Its likely you will get below error

[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:5.0.0.4389:sonar (default-cli) on project emapi: Unable to determine structure of project. Probably you use Maven Advanced Reactor Options with a broken tree of modules. -> [Help 1]

Steps To resolve error -

Please verify - Removed plugin old version from emapi\pom.xml

            <plugin> 
                <groupId>org.sonarsource.scanner.maven</groupId> 
                <artifactId>sonar-maven-plugin</artifactId> 
                <version>3.10.0.2594</version> 
            </plugin> 

Please verify - Removed below line from emapi\pom.xml

        <sonar.skip>true</sonar.skip> 

Run Sonar Scanner Maven

• Run

mvn clean verify -DskipUTs=false -DskipITs=false -Djacoco.skip=false jacoco:report jacoco:report-aggregate

• Run the maven goal sonar:sonar customized command given above.
  • Repeat run above command if getting error.

Sample output of mvn `sonar:sonar` e.g.

[INFO] --- sonar:5.0.0.4389:sonar (default-cli) @ emapi ---
[INFO] Java 17.0.10 Microsoft (64-bit)
[INFO] Windows 11 10.0 (amd64)
[INFO] User cache: C:\Users\Computer01\.sonar\cache
[INFO] Communicating with SonarQube Server 8.9.10.61524
[INFO] Load global settings
[INFO] Load global settings (done) | time=65ms
[INFO] Server id: BF41A1F2-AZTqdihJHPaIwNla_NrV
[INFO] User cache: C:\Users\Computer01\.sonar\cache
[INFO] Load/download plugins
[INFO] Load plugins index
[INFO] Load plugins index (done) | time=32ms
[INFO] Load/download plugins (done) | time=78ms
[INFO] Process project properties
[INFO] Process project properties (done) | time=6ms
[INFO] Execute project builders
[INFO] Execute project builders (done) | time=10ms
[INFO] Project key: emapi
[INFO] Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi
[INFO] Working dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\target\sonar
[INFO] Load project settings for component key: 'emapi'
[INFO] Load project settings for component key: 'emapi' (done) | time=22ms
[INFO] Load quality profiles
[INFO] Load quality profiles (done) | time=35ms
[INFO] Load active rules
[INFO] Load active rules (done) | time=1066ms
[WARNING] SCM provider autodetection failed. Please use "sonar.scm.provider" to define SCM of your project, or disable the SCM Sensor in the project settings.
[INFO] Indexing files...
[INFO] Project configuration:
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : app : dbgraphql'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\app\dbgraphql
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : app : dbrest'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\app\dbrest
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : app'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\app
[INFO]   Source paths: pom.xml
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : lib : base-app'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\lib\base-app
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : lib'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\lib
[INFO]   Source paths: pom.xml
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : testApi : dbgraphqlTest'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\testApi\dbgraphqlTest
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: src/test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : testApi : dbrestTest'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\testApi\dbrestTest
[INFO]   Source paths: pom.xml, src
[INFO]   Test paths: src/test
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi : testApi'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\testApi
[INFO]   Source paths: pom.xml
[INFO]   Excluded sources: **/*Test*/**
[INFO] Indexing files of module 'emapi'
[INFO]   Base dir: C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi
[INFO]   Source paths: pom.xml
[INFO]   Excluded sources: **/*Test*/**
[INFO] 140 files indexed
[INFO] 14 files ignored because of inclusion/exclusion patterns
[INFO] Quality profile for java: Sonar way
[INFO] Quality profile for xml: Sonar way
[INFO] ------------- Run sensors on module emapi : lib : base-app
[INFO] Load metrics repository
[INFO] Load metrics repository (done) | time=29ms
[INFO] Sensor JavaSquidSensor [java]
[INFO] Configured Java source version (sonar.java.source): 17
[INFO] JavaClasspath initialization
[INFO] JavaClasspath initialization (done) | time=10ms
[INFO] JavaTestClasspath initialization
[INFO] JavaTestClasspath initialization (done) | time=0ms
[INFO] Java Main Files AST scan
[INFO] 50 source files to be analyzed
[INFO] Load project repositories
[INFO] Load project repositories (done) | time=36ms
[INFO] 50/50 source files have been analyzed
[INFO] Java Main Files AST scan (done) | time=7050ms
[INFO] Java Test Files AST scan
[INFO] 6 source files to be analyzed
[INFO] 6/6 source files have been analyzed
[INFO] Java Test Files AST scan (done) | time=424ms
[INFO] Java Generated Files AST scan
[INFO] 0 source files to be analyzed
[INFO] 0/0 source files have been analyzed
[INFO] Java Generated Files AST scan (done) | time=0ms
[INFO] Sensor JavaSquidSensor [java] (done) | time=7694ms
[INFO] Sensor CSS Rules [cssfamily]
[INFO] No CSS, PHP, HTML or VueJS files are found in the project. CSS analysis is skipped.
[INFO] Sensor CSS Rules [cssfamily] (done) | time=0ms
[INFO] Sensor JaCoCo XML Report Importer [jacoco]
[INFO] 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
[INFO] Importing 1 report(s). Turn your logs in debug mode in order to see the exhaustive list.
[INFO] Sensor JaCoCo XML Report Importer [jacoco] (done) | time=157ms
[INFO] Sensor C# Project Type Information [csharp]
[INFO] Sensor C# Project Type Information [csharp] (done) | time=0ms
[INFO] Sensor C# Properties [csharp]
[INFO] Sensor C# Properties [csharp] (done) | time=0ms
[INFO] Sensor SurefireSensor [java]
[INFO] parsing [C:\Users\Computer01\Downloads\EmGenDir_johndoe_WS_50186\WS_50186\backend\spring-java\emapi\lib\base-app\target\surefire-reports]
[INFO] Sensor SurefireSensor [java] (done) | time=126ms
[INFO] Sensor JavaXmlSensor [java]
[INFO] 1 source file to be analyzed
[INFO] 1/1 source file has been analyzed
[INFO] Sensor JavaXmlSensor [java] (done) | time=110ms
[INFO] Sensor HTML [web]
[INFO] Sensor HTML [web] (done) | time=0ms
[INFO] Sensor XML Sensor [xml]
[INFO] 1 source file to be analyzed
[INFO] 1/1 source file has been analyzed
[INFO] Sensor XML Sensor [xml] (done) | time=110ms
[INFO] Sensor VB.NET Project Type Information [vbnet]
[INFO] Sensor VB.NET Project Type Information [vbnet] (done) | time=0ms
[INFO] Sensor VB.NET Properties [vbnet]
[INFO] Sensor VB.NET Properties [vbnet] (done) | time=0ms
...
[INFO] ------------- Run sensors on project
[INFO] Sensor Zero Coverage Sensor
[INFO] Sensor Zero Coverage Sensor (done) | time=0ms
[INFO] Sensor Java CPD Block Indexer
[INFO] Sensor Java CPD Block Indexer (done) | time=126ms
[INFO] SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.
[INFO] CPD Executor 11 files had no CPD blocks
[INFO] CPD Executor Calculating CPD for 56 files
[INFO] CPD Executor CPD calculation finished (done) | time=47ms
[INFO] Analysis report generated in 204ms, dir size=1 MB
[INFO] Analysis report compressed in 3944ms, zip size=458 KB
[INFO] Analysis report uploaded in 88ms
[INFO] ANALYSIS SUCCESSFUL, you can browse http://localhost:9000/dashboard?id=emapi
[INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
[INFO] More about the report processing at http://localhost:9000/api/ce/task?id=AZTquy79HPaIwNla_ScX
[INFO] Analysis total time: 19.891 s
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for emapi 1.0-SNAPSHOT:
[INFO]
[INFO] emapi .............................................. SUCCESS [ 22.294 s]
[INFO] emapi : lib ........................................ SKIPPED
[INFO] emapi : lib : base-app ............................. SKIPPED
[INFO] emapi : app ........................................ SKIPPED
[INFO] emapi : app : dbrest ............................... SKIPPED
[INFO] emapi : app : dbgraphql ............................ SKIPPED
[INFO] emapi : testApi .................................... SKIPPED
[INFO] emapi : testApi : dbrestTest ....................... SKIPPED
[INFO] emapi : testApi : dbgraphqlTest .................... SKIPPED
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
------------------------------------------------------------------------

• View SonarQube Analysis Report

e.g. at http://localhost:9000/dashboard?id=emapi

• View SonarQube Analysis Code Tab

e.g. at http://localhost:9000/code?id=emapi

Maven Dependency Analysis

Run Maven Dependency Analysis

Refer to generated Jenkinsfile in Spring Java

• backend\spring-java\emapi\Jenkinsfile

Refer to stage CI Run Apache Maven Dependency Analysis

• Run Maven Dependency Analysis

mvn org.apache.maven.plugins:maven-dependency-plugin:analyze-report

...
[INFO] <<< dependency:3.6.1:analyze-report (default-cli) < test-compile @ dbrest <<<
...
[INFO] --- dependency:3.6.1:analyze-report (default-cli) @ dbrest ---
...

• Report Location
  • Report at target/site/dependency-analysis.html
  • e.g. spring-java\emapi\app\dbrest\target\site\dependency-analysis.html

tests API Tests - TestNG, WebClient

• dbrestTest - Tests Run

• dbrestTest - Report Location

• dbrestTest - Report

• dbrestTest - Emailable Report

• dbgraphqlTest - Tests Run

• dbgraphqlTest - Report

• dbgraphqlTest - Emailable Report

• Aggregate Test report

API Logging

API Logging is included and available in emapi project.

e.g. API endpoint call - Query

curl -X 'GET' \
  'http://127.0.0.1:9080/emdbrest/tenant/Query?tenantId=3' \
  -H 'accept: application/hal+json'

Check API Logging

2024-10-02T16:57:21.469+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.w.f.CommonsRequestLoggingFilter      : Before request [GET /emdbrest/tenant/Query?tenantId=3]
2024-10-02T16:57:21.470+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.web.servlet.DispatcherServlet        : GET "/emdbrest/tenant/Query?tenantId=3", parameters={masked}
2024-10-02T16:57:21.471+05:30 DEBUG 35656 --- [io-9080-exec-10] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.example.emapi.app.Tenant.TenantDataRestController#TenantQuery(long)
2024-10-02T16:57:22.067+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Using 'application/hal+json', given [application/hal+json] and supported [application/json, application/*+json]
2024-10-02T16:57:22.067+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Writing [[Tenant [ tenantId = 3 ]]]
2024-10-02T16:57:22.071+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.web.servlet.DispatcherServlet        : Completed 200 OK
2024-10-02T16:57:22.072+05:30 DEBUG 35656 --- [io-9080-exec-10] o.s.w.f.CommonsRequestLoggingFilter      : API LOGGING: GET /emdbrest/tenant/Query?tenantId=3]

e.g. API endpoint call - Create with Payload

curl -X 'POST' \
  'http://127.0.0.1:9080/emdbrest/tenant/Create' \
  -H 'accept: application/hal+json' \
  -H 'Content-Type: application/json' \
  -d '  {
    "tenantId": 13,
    "customerId": 2,
    "tenantName": "Manipal - Pune Branch 2",
    "beginDate": "2024-09-19",
    "minOrderQuantity": 10,
    "customizationRequired": 1,
    "custMicrosvcType": "synchronous-webclient",
    "customizationService": "tenant-2-validation-service",
    "customizationPayload": "order_data"
  }'

Check API Logging with Payload

2024-10-02T17:06:11.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.w.f.CommonsRequestLoggingFilter      : Before request [POST /emdbrest/tenant/Create]
2024-10-02T17:06:11.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.web.servlet.DispatcherServlet        : POST "/emdbrest/tenant/Create", parameters={}
2024-10-02T17:06:11.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.example.emapi.app.Tenant.TenantDataRestController#TenantCreate(Tenant)
2024-10-02T17:06:11.249+05:30 DEBUG 35656 --- [nio-9080-exec-7] m.m.a.RequestResponseBodyMethodProcessor : Read "application/json;charset=UTF-8" to [Tenant [ tenantId = 13 ]]
2024-10-02T17:06:11.488+05:30  INFO 35656 --- [nio-9080-exec-7] c.e.e.a.Tenant.TenantDataRestController  : Tenant Begin Create Record For [Tenant [ tenantId = 13 ]] 
2024-10-02T17:06:11.746+05:30  INFO 35656 --- [nio-9080-exec-7] c.e.emapi.app.Tenant.TenantService       : Tenant Created Record [Tenant [ tenantId = 13 ]] 
2024-10-02T17:06:12.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Using 'application/hal+json', given [application/hal+json] and supported [application/json, application/*+json]
2024-10-02T17:06:12.229+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Writing [Tenant [ tenantId = 13 ]]
2024-10-02T17:06:12.230+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.web.servlet.DispatcherServlet        : Completed 201 CREATED
2024-10-02T17:06:12.231+05:30 DEBUG 35656 --- [nio-9080-exec-7] o.s.w.f.CommonsRequestLoggingFilter      : API LOGGING: POST /emdbrest/tenant/Create, payload=  {
    "tenantId": 13,
    "customerId": 2,
    "tenantName": "Manipal - Pune Branch 2",
    "beginDate": "2024-09-19",
    "minOrderQuantity": 10,
    "customizationRequired": 1,
    "custMicrosvcType": "synchronous-webclient",
    "customizationService": "tenant-2-validation-service",
    "customizationPayload": "order_data"
  }]

Note:

• API Payload Logging is set for max length 10000. Adjust setting or disable it in file:
  • emapi\lib\base-app\src\main\java\com\example\emapi\app\EmApiRequestLoggingFilterConfig.java

Security - OAuth2 with Keycloak

caution

• Keycloak server needs to be accessible and running.
• Run Keycloak in Docker
• Using template for it, More info
  • Tutorial 3 : EasyManage Templates

• Run Keycloak in Docker

Build emapi project with Project Configuration,

• Security Configuration :

  • OAuth2 [Authorization Server]

• Compile and Run project

• Verify APIs security

• Open Swagger API explorer and test API endpoints

  • http://127.0.0.1:9080/swagger-ui/index.html
  • Note: Swagger UI endpoints displaying is permitted without security.

• Test APIs without auth token

curl -X 'GET' \
  'http://127.0.0.1:9080/emdbrest/erp_product/ViewAll' \
  -H 'accept: application/hal+json'

Server response:

Code    Details
401   Error: response status is 401

Log shows:

2024-10-02T17:50:42.428+05:30 DEBUG 36388 --- [nio-9080-exec-6] o.s.security.web.FilterChainProxy        : Securing GET /emdbrest/erp_product/ViewAll
2024-10-02T17:50:42.428+05:30 DEBUG 36388 --- [nio-9080-exec-6] o.s.s.w.a.AnonymousAuthenticationFilter  : Set SecurityContextHolder to anonymous SecurityContext
2024-10-02T17:50:42.428+05:30 DEBUG 36388 --- [nio-9080-exec-6] o.s.s.w.a.i.FilterSecurityInterceptor    : Failed to authorize filter invocation [GET /emdbrest/erp_product/ViewAll] with attributes [hasRole('ROLE_USER')]
2024-10-02T17:50:42.429+05:30 DEBUG 36388 --- [nio-9080-exec-6] o.s.s.w.s.HttpSessionRequestCache        : Saved request http://127.0.0.1:9080/emdbrest/erp_product/ViewAll?continue to session

• Get auth token: Giving client_secret and password values

• Note: Token expires in 60 sec by default

curl -d "client_id=login-app&client_secret=<client_secret>&username=emUser&password=<password>&grant_type=password" http://127.0.0.1:8180/realms/emapi/protocol/openid-connect/token

Representative Output: Trimmed for displaying here ...

{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJNaU5KUmhtODZiYlZkUk82bGdiVC1EM3c5WEM4U2pkdnJBRndZYkVBaWFJIn0
...
rTmF-Ld1LMiksm8QBBsYGaZOKcWIiXXRuGhGD4RahbhnU6ckQoGD1LPnLNUHLwrTJT7YUW_1dCDMi95231q_MP4XFpmzCGpxBx4tRFmPf2Ag","expires_in":300,"refresh_expires_in":1800,"refresh_token":"eyJhbGciOiJIUzUxMiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIwN2UwZDk0O
...
kmU-2OxA","token_type":"Bearer","not-before-policy":0,"session_state":"81f96ffc-6a5f-484c-878e-d4456b3af966","scope":"email profile"}

Note auth token to use in curl below.

• Test APIs with auth token

curl -H "Accept: application/json" -H "Authorization: Bearer <token>" "http://127.0.0.1:9080/emdbrest/erp_product/Query?productId=1"

Server response:

[{"productId":1,"productName":"string new","productCategory":"string new","primarySupplier":"string","productDesc":"string","productPicture":null}]

Log shows:

2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.security.web.FilterChainProxy        : Securing GET /emdbrest/erp_product/Query?productId=1
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.s.o.s.r.a.JwtAuthenticationProvider  : Authenticated token
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] .s.r.w.a.BearerTokenAuthenticationFilter : Set SecurityContextHolder to JwtAuthenticationToken [Principal=org.springframework.security.oauth2.jwt.Jwt@82025aa3, Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=127.0.0.1, SessionId=null], Granted Authorities=[ROLE_USER]]
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.s.w.a.i.FilterSecurityInterceptor    : Authorized filter invocation [GET /emdbrest/erp_product/Query?productId=1] with attributes [hasRole('ROLE_USER')]
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.security.web.FilterChainProxy        : Secured GET /emdbrest/erp_product/Query?productId=1
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.w.f.CommonsRequestLoggingFilter      : Before request [GET /emdbrest/erp_product/Query?productId=1]
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.web.servlet.DispatcherServlet        : GET "/emdbrest/erp_product/Query?productId=1", parameters={masked}
2024-10-02T18:11:30.987+05:30 DEBUG 36388 --- [nio-9080-exec-9] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.example.emapi.app.ErpProduct.ErpProductDataRestController#ErpProductQuery(long)
2024-10-02T18:11:31.507+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Using 'application/json', given [application/json] and supported [application/json, application/*+json]
2024-10-02T18:11:31.507+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.w.s.m.m.a.HttpEntityMethodProcessor  : Writing [[ErpProduct [ productId = 1 ]]]
2024-10-02T18:11:31.507+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.web.servlet.DispatcherServlet        : Completed 200 OK
2024-10-02T18:11:31.507+05:30 DEBUG 36388 --- [nio-9080-exec-9] o.s.w.f.CommonsRequestLoggingFilter      : API LOGGING: GET /emdbrest/erp_product/Query?productId=1]

If token expired, The logs show

2024-10-02T18:08:11.335+05:30 DEBUG 36388 --- [nio-9080-exec-3] o.s.s.oauth2.jwt.JwtTimestampValidator   : Jwt expired at 2024-10-02T12:34:29Z
2024-10-02T18:08:11.335+05:30 DEBUG 36388 --- [nio-9080-exec-3] o.s.s.o.s.r.a.JwtAuthenticationProvider  : Failed to authenticate since the JWT was invalid

Security - OAuth2 with Keycloak With OAuth2 Client

• Take-home Exercises
  • Implement Security Requirements for below.
  • OAuth2 [Authorization Server + OAuth2 Client For Web Apps]
  • OAuth2 [Authorization Server + OAuth2 Client For BFF Auth APIs]

Redis cache

caution

• Redis server needs to be accessible and running.
• Run Redis in Docker
• Using template for it, More info
  • Tutorial 3 : EasyManage Templates

• Run Redis in Docker

Build emapi project with Project Configuration,

• Redis Cache : Include [✅] Enable [✅]

• Enable Swagger UI to display Request Duration Swagger UI has the display Request Duration parameter to show how long a request takes. Enable it.

  • Edit : emapi\app\dbrest\src\main\resources\application.properties
  • Add line springdoc.swagger-ui.display-request-duration=true

• Compile and Run project

• Verify APIs response times

• Open Swagger API explorer and test API endpoints

  • http://127.0.0.1:9080/swagger-ui/index.html

• Test APIs : 1st call

http://127.0.0.1:9080/emdbrest/erp_customer/Query?customerId=5

Server response in:
Request Duration 
1045 ms

• Test APIs : 1st call

• Test APIs : 2nd call - Which will use redis cached data!

http://127.0.0.1:9080/emdbrest/erp_customer/Query?customerId=5

Server response in:
Request Duration 
104 ms

• Test APIs : 2nd call

Serverless

• Covered in tutorial : Serverless

Logging - ELK stack

caution

• ELK Stack consists of Elasticsearch, Logstash, and Kibana
• ELK servers needs to be accessible and running.
• Run ELK in Docker
• Using template for it, More info
  • Tutorial 3 : EasyManage Templates

• Run ELK in Docker

Build emapi project with Project Configuration,

• Logging - ELK : Include [✅] Enable [✅]

Add custom logs for observability and monitoring, e.g.:

    logger.info(String.format("ErpCustomer list size=[%s] firstCustomerId=[%s] firstCustomerName=[%s]", ErpCustomerList.size(), ErpCustomerList.get(0).getCustomerId(), ErpCustomerList.get(0).getName()));

• Compile and Run project
• Verify APIs
• Open Swagger API explorer and test API endpoints
  • http://127.0.0.1:9080/swagger-ui/index.html

Log shows:

logdate=(2024-10-02T19:25:05,046) thread=(main)) level=(DEBUG) loggerclass=(org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver) message=(ControllerAdvice beans: 2 @ExceptionHandler, 1 ResponseBodyAdvice)
logdate=(2024-10-02T19:25:05,091) thread=(main)) level=(DEBUG) loggerclass=(org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver) message=(ControllerAdvice beans: 2 @ExceptionHandler, 0 ResponseBodyAdvice)
logdate=(2024-10-02T19:25:05,593) thread=(main)) level=(INFO) loggerclass=(com.example.emapi.app.EmDbRestAppRestSpringApp) message=(Started EmDbRestAppRestSpringApp in 9.93 seconds (process running for 11.36))

• Test APIs : Make Create calls, with "productId" = 800, 801, 802

curl -X 'POST' \
  'http://127.0.0.1:9080/emdbrest/erp_product/Create' \
  -H 'accept: application/hal+json' \
  -H 'Content-Type: application/json' \
  -d '{
  "productId": 802,
  "productName": "Product 802",
  "productCategory": "Vitamin",
  "primarySupplier": "Cipla"
}'

View Logs in Kibana

Access from Kibana UI: http://localhost:5601

• Configure index pattern: logstash-*
• Visualize : Create Visualization
  • View

DevOps

• DevOps CI/CD pipelines for build, verify, deploy - using Jenkins.

CI/CD - Continuous Integration/Continuous Delivery

Jenkins

Refer to generated Jenkinsfile in Spring Java project.

• backend\spring-java\emapi\Jenkinsfile

GitLab CI/CD

Refer to generated .gitlab-ci.yml in Spring Java project.

• backend\spring-java\emapi\.gitlab-ci.yml

DevSecOps

DevSecOps is an extension of DevOps that integrates security into the entire software development process, specifically into CI/CD pipeline.

DevSecOps Tools

For Backend - Java Spring Boot

SonarQube

SonarScanner used for assessing code quality, analyze the code including

• Static Code Analysis
• Security Analysis

• Please refer to SonarScanner - SonarQube

Maven Dependency Analysis

• Please refer to Maven Dependency Analysis

OWASP dependency check Maven

Note: 1st Run >20Min

• Please see Jenkin pipeline OWASP dependency check stage in emapi\Jenkinsfile

Sonatype Lifecycle

• Please refer to: EasyManage: Backend Templates
  • Locate in generated code folder ...\resources\templates\backend\spring-java
  • Readme DevSecOps: backend\spring-java\DevSecOps\README-DevSecOps.md
  • Readme Sonatype Lifecycle: backend\spring-java\DevSecOps\sonatype\README-sonatype.md

Please see next section for a brief overview on Sonatype Nexus Scan.

• Sonatype Nexus Scan

Run Sonatype Nexus Scan with Nexus IQ Server in docker

Install/Setup and Run Nexus IQ Server in docker

docker run -d -p 8070:8070 -p 8071:8071 --name nexus-iq-server sonatype/nexus-iq-server

• Login to http://localhost:8070/
  • login: admin / admin123
  • Once running, the IQ Server product license must be installed.

Run Sonatype Nexus Scan

• Run the maven goal evaluate

mvn package com.sonatype.clm:clm-maven-plugin:evaluate -Dclm.additionalScopes=test,provided -Dclm.applicationId=emapi -Dclm.serverUrl=http://localhost:8070 -Dclm.username=admin -Dclm.password=admin123

Sample output of mvn `evaluate` e.g.

[INFO] --- clm-maven-plugin::evaluate (default-cli) @ line-comm-03 ---
[INFO] Starting scan...
...
[INFO] Evaluating policies on http://localhost:8070 ...
[ERROR] Sonatype IQ reports policy 'Security-Critical' failing for 
  component 'org.apache.logging.log4j:log4j-core:2.9.0' with hash '052f6548ae1688e126c2' due to 
    constraint 'Critical risk CVSS score': 
      Security Vulnerability Severity >= 9 because: Found security vulnerability CVE-2021-44228 with severity >= 9 (severity = 10.0)
[INFO] ------------------------------------------------------------------------
[INFO] Policy Action: Failure
[INFO] Number of components affected: 1 critical, 0 severe, 0 moderate
[INFO] Number of open policy violations: 2 critical, 0 severe, 1 moderate
[INFO] Number of grandfathered policy violations: 0
[INFO] Number of components evaluated: 5
[INFO] The detailed report can be viewed at: http://localhost:8070/ui/links/application/local-iq-app/report/755d19e970fd491ca2e23f21bea35d58
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
------------------------------------------------------------------------

• View Result Of Sonatype Nexus Scan

  • Please see details in output log:
    • Policy Action: None | Warn | Failure
    • The detailed report can be viewed at: Given Link