# Deployment Guide β€” NOVA Invoice Suite > **🌐 Portale NOVA** β€” questo doc Γ¨ anche disponibile in versione HTML premium nel [Documentation Portal](_portal/viewer.html?path=DEPLOYMENT_GUIDE.md). Sizing rapido: [Premium Wizard](sizing-wizard/premium.html). Lifecycle interattivo: [Visualizer](lifecycle-visualizer/index.html). > Istruzioni step-by-step per deployment su SAP BTP Cloud Foundry, BTP Kyma, e on-premise. > Include attivitΓ  su Global Account, Subaccount, IAS, role collections, service plans, Terraform IaC. > > **Lettori**: DevOps / Implementation Consultant / SAP Basis. > **Aggiornato**: 2026-04-28 (10 CDS profiles, Terraform IaC 3 providers, 4 CI/CD modes incl. cTMS+Piper). > > **Diagrammi deployment**: > - [`architecture/deployment-btp-cf-full-sap.svg`](architecture/deployment-btp-cf-full-sap.svg) β€” BTP CF full SAP stack > - [`architecture/deployment-btp-kyma-hybrid.svg`](architecture/deployment-btp-kyma-hybrid.svg) β€” Kyma + BTP hybrid (PG/Redis in-cluster + AI Core) > - [`architecture/deployment-kyma-onprem-full-foss.svg`](architecture/deployment-kyma-onprem-full-foss.svg) β€” On-prem k3d full FOSS (Keycloak/Ollama) ## Indice 1. [Decision matrix β€” quale target scegliere](#1-decision-matrix--quale-target-scegliere) 2. [Pre-requisiti comuni](#2-pre-requisiti-comuni) 2-bis. [Infrastructure-as-Code (Terraform) β€” opzionale ma raccomandato](#2-bis-infrastructure-as-code-terraform--opzionale-ma-raccomandato) 3. [BTP Global Account β€” setup iniziale](#3-btp-global-account--setup-iniziale) 4. [BTP Subaccount β€” entitlements e service plans](#4-btp-subaccount--entitlements-e-service-plans) 5. [SAP IAS β€” configurazione applicazione + groups](#5-sap-ias--configurazione-applicazione--groups) 6. [BTP CF deployment β€” step-by-step](#6-btp-cf-deployment--step-by-step) 7. [BTP Kyma deployment β€” step-by-step](#7-btp-kyma-deployment--step-by-step) 8. [On-premise (k3d/k3s) deployment](#8-on-premise-k3dk3s-deployment) 9. [Post-deploy verification](#9-post-deploy-verification) 10. [Update / rollback](#10-update--rollback) 10-bis. [GitHub Actions CI/CD pipelines](#10-bis-github-actions-cicd-pipelines) 11. [Troubleshooting deploy](#11-troubleshooting-deploy) --- ## 1. Decision matrix β€” quale target scegliere ```mermaid flowchart TD Start{Customer ha BTP
subscription?} -->|No| OnPrem[On-premise
k3d / k3s] Start -->|Sì| Subscription{Tipo subscription?} Subscription -->|Free Tier / Trial| Kyma[BTP Kyma
k8s] Subscription -->|Standard / Enterprise| HasS4{Customer ha
SAP managed services?} HasS4 -->|Sì BPA, AI Core, ANS, ecc.| CF[BTP CF
production] HasS4 -->|No, vuole minimal| Kyma Kyma --> DBChoice{DB choice?} DBChoice -->|Postgres managed| KymaPG[Kyma + PostgreSQL
k8s profile] DBChoice -->|HANA on-prem| KymaHANA[Kyma + HANA
k8s-hana profile] OnPrem --> OnPremChoice{Auth?} OnPremChoice -->|Keycloak| K3d[k3d + Keycloak
k8s-onprem profile] OnPremChoice -->|XSUAA on-prem| OnPremCF[CF on-prem
onprem profile] ``` | Target | Profile | Deploy time | Use case | |---|---|---|---| | **BTP CF** | `production` | 30-45 min | Customer enterprise BTP-native, vuole servizi managed | | **BTP Kyma** | `k8s` | 20-30 min | Free tier OR customer cloud-native, vuole flessibilitΓ  K8s | | **BTP Kyma + HANA** | `k8s-hana` | 30-40 min | Customer con HANA on-prem esistente | | **On-prem k3d** | `k8s-onprem` | 60-90 min | Air-gapped, customer data sovereignty | ## 2. Pre-requisiti comuni ### Tools developer machine ```bash node --version # 22.x npm --version # 10+ cf --version # CF CLI 8+ (BTP CF target) kubectl version --client # 1.27+ (Kyma/k8s target) helm version # 3.12+ (Kyma) mbt --version # 1.2+ (BTP CF MTA build) docker --version # 24+ gh --version # GitHub CLI per workflow trigger ``` Install missing tools: ```bash # CF CLI brew tap cloudfoundry/tap brew install cf-cli@8 # kubectl + helm brew install kubectl helm # mbt (Multi-Target Application Build Tool) npm install -g mbt @sap/cds-dk # GitHub CLI brew install gh ``` ### Repository access ```bash git clone https://github.com//passive-invoice-mgmt.git cd passive-invoice-mgmt npm ci --legacy-peer-deps ``` ### Verify build ```bash npx cds compile srv/orchestratorSrv.cds -s OrchestratorService --to edmx # expected: no errors npm test # expected: ~117 suite / 1417 tests passing ``` --- ## 2-bis. Infrastructure-as-Code (Terraform) β€” opzionale ma raccomandato Le sezioni Β§3-Β§7 sotto descrivono il setup **manuale** via BTP Cockpit + UI. Per automatizzare entitlements, subaccount, role collections, IAS, Cloud Connector e destinations Γ¨ disponibile uno stack **Terraform** in [terraform/](../terraform/) β€” usato dal team NOVA per onboarding multi-customer. ### 2-bis.1 Quando usarlo | Scenario | Manuale (Β§3-Β§7) | Terraform | |---|---|---| | Demo/pilot single-customer | βœ… | overkill | | Onboarding ricorrente customer enterprise | ⚠️ error-prone | βœ… | | Multi-region deployment (eu10 + us10) | ❌ drift garantito | βœ… | | Compliance audit "infra reproducible" | ❌ | βœ… | ### 2-bis.2 Stack provider Definito in [terraform/versions.tf](../terraform/versions.tf) (separato da main.tf per chiarezza): | Provider | Source | Versione | Scopo | |---|---|---|---| | `btp` | `SAP/btp` | ~> 1.21 | Subaccount, entitlements, service instances, role collections, destinations | | `sci` | `SAP/sap-cloud-identity-services` | ~> 0.5 | IAS application, OIDC, role groups, users | | `scc` | `SAP/scc` | ~> 1.3 | Cloud Connector subaccount registration, system mapping, OData paths | Versioni allineate al 2026-04-27 contro [registry.terraform.io](https://registry.terraform.io/providers/SAP/). Bump quarterly o su security fix. ### 2-bis.3 Moduli inclusi | Modulo | Path | Risorse | Quando usarlo | |---|---|---|---| | **ias** | [terraform/modules/ias/](../terraform/modules/ias/) | OIDC application + 4 role groups (Viewer/Worker/Approver/Admin) + initial admin user | `enable_ias=true` | | **scc** | [terraform/modules/scc/](../terraform/modules/scc/) | Cloud Connector subaccount registration + S/4HANA system mapping | `enable_scc=true` (on-premise S/4) | | **destinations** | [terraform/modules/destinations/](../terraform/modules/destinations/) | 12 BTP destinations (S/4 Business Partner, PO, Supplier Invoice, ElectronicDocFile, ecc.) | `enable_destinations=true` | ### 2-bis.4 Variabili input principali Definite in [terraform/variables.tf](../terraform/variables.tf) (40+ vars). Quelle obbligatorie: ```hcl globalaccount_subdomain = "your-ga-subdomain" # da BTP Cockpit client_name = "acme" # identifier per naming admin_email = "admin@example.com" # initial admin Role Collection region = "eu10" # BTP region runtime = "kyma" # "kyma" | "cf" db_type = "postgresql-incluster" # "postgresql-incluster" | "postgresql-btp" | "hana" ``` Variabili optional services (default `false`, abilitarle quando customer richiede): | Variabile | Effetto | Adapter abilitato | |---|---|---| | `enable_ias` | Crea IAS app + role groups | OIDC auth strategy | | `enable_scc` | Registra Cloud Connector | S/4 on-prem connectivity | | `enable_destinations` | Crea 12 BTP destinations | S/4 Cloud connectivity | | `enable_alert_notification` | Provisiona ANS | `notification.adapter=ans` | | `enable_cloud_logging` | Provisiona Cloud Logging | telemetry export | | `enable_enterprise_messaging` | Provisiona Event Mesh | `messaging.adapter=eventmesh` | | `enable_job_scheduling` | Provisiona Job Scheduler | `JOB_SERVICE_TOKEN` consumer | | `enable_dox` | Provisiona Document Information Extraction | `extraction.adapter=dox` | | `enable_auditlog` | Provisiona BTP Audit Log | `audit.adapter=btp` | | `enable_bpa` | Provisiona Build Process Automation | `workflow.adapter=bpa` | | `enable_integration_suite` | Provisiona Integration Suite | `_VIA_IS` routing strategy | | `enable_ai_core` | Provisiona AI Core (extended plan) | `AI_ADAPTER=genai-hub` | | `enable_workzone` | Provisiona Build Work Zone | FLP hosting + Key User Adaptation | | `enable_ui5_flexibility` | Provisiona UI Flexibility | `FlexKeyUser` role utilizzabile | | `enable_responsibility_management` | Provisiona Responsibility Mgmt | `responsibility.adapter=rm` | | `enable_redis` | Provisiona Redis (CF only) | `cache.adapter=redis` | | `enable_cloud_alm` | Provisiona Cloud ALM | `monitoring.adapter=cloud-alm` | | `enable_autoscaler` | Provisiona Application Autoscaler (CF only) | HPA equivalente CF | | `enable_custom_domain` | Provisiona Custom Domain plan | DNS personalizzato | ### 2-bis.5 State backend ⚠️ **CRITICAL**: il file `terraform.tfstate` contiene credenziali S/4HANA in plaintext. **Configurare un remote backend prima del primo `apply` su production**. Vedi [terraform/main.tf:21-42](../terraform/main.tf) per i 2 backend supportati: **Opzione A β€” AWS S3 + DynamoDB lock**: ```hcl backend "s3" { bucket = "nova-terraform-state" key = "btp/terraform.tfstate" region = "eu-central-1" dynamodb_table = "nova-terraform-locks" encrypt = true } ``` **Opzione B β€” Terraform Cloud (free tier)**: ```hcl cloud { organization = "nova-invoice-suite" workspaces { name = "btp-production" } } ``` Il file di esempio Γ¨ in [terraform/backend.tf.example](../terraform/backend.tf.example). ### 2-bis.6 Workflow per nuovo customer ```bash # 1. Setup wizard interattivo (genera tfvars per il customer) chmod +x terraform/configure.sh ./terraform/configure.sh # β†’ Scegli runtime, db, optional services, IAS tenant # β†’ Output: terraform/clients/.tfvars # 2. Inizializza terraform (download providers + state backend) cd terraform terraform init # 3. Set secrets via env vars (NO commit in tfvars!) export TF_VAR_s4hana_password="..." export SCI_USERNAME="..." export SCI_PASSWORD="..." export SCC_USERNAME="..." export SCC_PASSWORD="..." # 4. Plan + apply terraform plan -var-file="clients/acme.tfvars" terraform apply -var-file="clients/acme.tfvars" # 5. Output β†’ GitHub Secrets per pipeline deploy terraform output -json > /tmp/tf-output.json # Estrai kyma_dashboard_url, subaccount_id, xsuaa_instance_id per env "production" ``` ### 2-bis.7 Best practice - **Branch strategy**: 1 branch terraform per customer (`tf/acme`, `tf/foo`) per evitare drift state durante apply concorrenti - **Tfvars naming**: sempre `clients/.tfvars`, mai inline su `terraform apply -var=...` (perde reproducibility) - **Secrets**: env vars `TF_VAR_*`, MAI nel tfvars; per CI usa GitHub Actions secrets + `.github/workflows/_deploy-customer.yml` - **Drift detection**: `terraform plan` settimanale (cron) per detect manual changes via BTP Cockpit - **Destroy con cautela**: `terraform destroy` su subaccount production = perdita totale dati. Sempre `terraform plan -destroy` prima ### 2-bis.8 Limitazioni note - **Kyma in-cluster resources** (PostgreSQL Bitnami, MinIO, Redis): NON gestiti da terraform. Vanno deployati via Helm/kubectl post `terraform apply`. Vedi Β§7 per workflow Kyma. - **APIRule + ConfigMap + Secret K8s**: gestiti via `kubectl apply` nella pipeline `.github/workflows/deploy.yml`, NON da terraform. - **xs-security.json**: la version "in-line" nel main.tf differisce da [xs-security.json](../xs-security.json) β€” quest'ultima Γ¨ la SSOT runtime. Il main.tf serve solo a creare le 4 role collections; per scope/role-template completi (15 ruoli) vedi xs-security.json. ### 2-bis.9 Test connessione ```bash # Verify BTP credentials btp login --user $BTP_USERNAME # Verify SCI (IAS) credentials curl -u "$SCI_USERNAME:$SCI_PASSWORD" https:///scim/v2/Users?count=1 # Verify Terraform plan idempotency (no changes after apply) terraform plan -var-file="clients/.tfvars" # Expected: "No changes. Your infrastructure matches the configuration." ``` ### 2-bis.10 Riferimenti **Root files**: - [terraform/README.md](../terraform/README.md) β€” quickstart + indice - [terraform/versions.tf](../terraform/versions.tf) β€” provider version constraints - [terraform/main.tf](../terraform/main.tf) β€” root configuration (~25 risorse) - [terraform/variables.tf](../terraform/variables.tf) β€” 50+ input vars con validation rules - [terraform/outputs.tf](../terraform/outputs.tf) β€” output per CI integration - [terraform/backend.tf.example](../terraform/backend.tf.example) β€” template backend (4 opzioni) - [terraform/configure.sh](../terraform/configure.sh) β€” wizard interattivo con input validation **Guides** (operational deep-dives): - [terraform/guides/STATE_MANAGEMENT.md](../terraform/guides/STATE_MANAGEMENT.md) β€” backend setup S3/TF Cloud/Azure/GCS - [terraform/guides/DRIFT_DETECTION.md](../terraform/guides/DRIFT_DETECTION.md) β€” `plan -refresh-only` + GHA workflow - [terraform/guides/MIGRATION_WITH_EXPORTER.md](../terraform/guides/MIGRATION_WITH_EXPORTER.md) β€” `btptf` import legacy account - [terraform/guides/SENSITIVE_DATA.md](../terraform/guides/SENSITIVE_DATA.md) β€” gestione credentials state/env/CI **Examples** (tfvars sample): - [terraform/examples/basic/](../terraform/examples/basic/) β€” Kyma + PostgreSQL incluster (zero servizi opzionali) - [terraform/examples/production/](../terraform/examples/production/) β€” CF + HANA Cloud + tutti i servizi - [terraform/examples/migration/](../terraform/examples/migration/) β€” workflow btptf importing existing account **Module READMEs**: - [terraform/modules/ias/README.md](../terraform/modules/ias/README.md) β€” OIDC app + 4 IAS groups - [terraform/modules/scc/README.md](../terraform/modules/scc/README.md) β€” Cloud Connector subaccount + system mapping - [terraform/modules/destinations/README.md](../terraform/modules/destinations/README.md) β€” 12 destinations S/4HANA **Esempi tfvars committed**: - [terraform/clients/demo.tfvars](../terraform/clients/demo.tfvars) β€” esempio Kyma free + PostgreSQL in-cluster - [terraform/examples/basic/terraform.tfvars](../terraform/examples/basic/terraform.tfvars) β€” minimal pilot - [terraform/examples/production/terraform.tfvars](../terraform/examples/production/terraform.tfvars) β€” full enterprise stack **Tool ufficiali SAP**: - [terraform-provider-btp](https://registry.terraform.io/providers/SAP/btp/latest/docs) β€” 44 resources, 60+ data sources, 5 functions - [terraform-provider-sap-cloud-identity-services](https://registry.terraform.io/providers/SAP/sap-cloud-identity-services/latest/docs) β€” IAS provider (beta) - [terraform-provider-scc](https://registry.terraform.io/providers/SAP/scc/latest/docs) β€” Cloud Connector provider - [terraform-exporter-btp (`btptf`)](https://sap.github.io/terraform-exporter-btp/) β€” migration tool legacy account β†’ IaC - [terraform-landingpage-for-btp](https://sap-docs.github.io/terraform-landingpage-for-btp/) β€” hub navigazione SAP Terraform --- ## 3. BTP Global Account β€” setup iniziale > **Da fare una volta per global account.** ### 3.1 Accesso BTP Cockpit 1. Login `https://cockpit.btp.cloud.sap` con email + IAS authentication 2. Naviga a **Global Account β†’ ** ### 3.2 Verify quota e regions | Service | Quota richiesta minima | Region consigliata | |---|---|---| | Cloud Foundry runtime | 10 GB memoria | `eu10` (Frankfurt) o region piΓΉ vicina al customer | | Kyma runtime | 1 cluster (3 nodi 4vCPU/16GB) | id. | | HANA Cloud | 1 instance (HDI) | id. | | Audit Log | 1 (premium per long retention) | id. | | SAP IAS tenant | 1 tenant | (region-specific) | In **Global Account β†’ Entitlements β†’ Configure Entitlements**, verifica che le seguenti voci siano disponibili (senza fare assignment ancora): - Cloud Foundry Runtime (almeno `application` plan) - Kyma Runtime (`free` o `standard`) - SAP HANA Cloud (`hana` plan) - Authorization and Trust Management Service (XSUAA) - Destination service (`lite` o `standard`) - HTML5 Application Repository (`app-host` + `app-runtime`) - Audit Log Service (`oauth2` plan) - SAP Cloud Logging (`standard` plan) - Alert Notification Service - Job Scheduling Service (`standard`) - Document Information Extraction (`default`) - Enterprise Messaging (`default-cf` o `mesh-default`) - Redis on Cloud Foundry (`development` o `standard`) β€” solo CF - Application Autoscaler (`standard`) β€” solo CF - SAP Build Workzone (`standard`) β€” opzionale per Launchpad - UI5 Flexibility for Key Users (`standard`) β€” opzionale per personalization - SAP Responsibility Management (`standard`) β€” opzionale > Free tier limits: vedi `https://discovery-center.cloud.sap/serviceCatalog`. Alcuni servizi hanno solo plan a pagamento. ### 3.3 Trust configuration con SAP IAS 1. Global Account β†’ **Security β†’ Trust Configuration** 2. Verifica che IAS tenant Γ¨ configurato come default IdP (oppure aggiungi manualmente con `Establish Trust`) 3. Annota tenant URL (es. `https://.accounts.ondemand.com`) β€” useremo dopo ### 3.4 Custom domain (production) Per production con dominio custom (es. `nova.example.com`): 1. **Global Account β†’ Custom Domain Service β†’ Subscribe** 2. Carica certificato wildcard (`*.example.com`) tramite **Custom Domain Manager** 3. DNS CNAME `*.example.com` β†’ `*.cfapps..hana.ondemand.com` ## 4. BTP Subaccount β€” entitlements e service plans > **Per ogni customer (subaccount per customer = best practice multi-tenant).** ### 4.1 Crea Subaccount In Global Account: 1. **Sub Accounts β†’ Create** 2. Nome: `nova-invoice-{customer}-{env}` (es. `nova-invoice-acme-prod`) 3. Region: stessa Global Account 4. Provider: `Cloud Foundry` o `Kyma` (multi-runtime ok) ### 4.2 Assign entitlements Per **BTP CF target**: ``` Subaccount β†’ Entitlements β†’ Edit - Cloud Foundry Runtime: 10 GB - SAP HANA Cloud: 1 instance hana plan - Authorization and Trust Management: 1 application plan - Destination: 1 lite plan - HTML5 Application Repository: - app-host: 1 plan - app-runtime: 5 plan - Audit Log Service: 1 oauth2 (premium se long retention) - SAP Cloud Logging: 1 standard - Alert Notification: 1 standard - Job Scheduling: 1 standard - Document Information Extraction: 1 default - Enterprise Messaging: 1 default-cf - Redis on Cloud Foundry: 1 development - Application Autoscaler: 1 standard - AI Core: 1 standard (opzionale) - Cloud Integration Automation Service (CIAS): 1 standard ``` **Opzionale β€” per code transport modes 2/3/4** (vedi Β§10-bis.9): ``` - Cloud Transport Management: 1 export plan # MODE 2/3/4 - Continuous Integration & Delivery: 1 standard # MODE 3 only ``` Setup dettagliato del service instance + destination cTMS: vedi [docs/ctms-integration.md Β§"Service instance + destination"](ctms-integration.md#service-instance--destination). Per **Kyma target**: come sopra MENO Redis/Autoscaler (in-cluster) e PIU' ``` - Kyma Runtime: 1 standard plan (cluster managed) - SAP IAS Application Service Binding: 1 ``` Save β†’ wait for provisioning (~5min). ### 4.3 Enable Cloud Foundry / Kyma environment #### CF environment 1. Subaccount β†’ **Cloud Foundry Environment β†’ Enable** 2. Name org: `-org` 3. Wait ~3min 4. Click **Create Space** β†’ name `production` > Spaces: convention `dev`, `qa`, `production`. Quota separata per space. #### Kyma environment 1. Subaccount β†’ **Kyma Environment β†’ Enable** 2. Plan: `standard` (3 nodi default) 3. Cluster name: `-kyma` 4. Wait ~10min per provisioning 5. Download kubeconfig: **Kyma Environment β†’ Kubeconfig URL β†’ Download** ### 4.4 Service instances Vedi [Β§6](#6-btp-cf-deployment--step-by-step) (CF) o [Β§7](#7-btp-kyma-deployment--step-by-step) (Kyma) per le istanze concrete da creare. ### 4.5 Role Collections Per ogni ruolo applicativo crea una **Role Collection** in **Subaccount β†’ Security β†’ Role Collections**: | Role Collection | Roles incluse | Assigned to (typical) | |---|---|---| | `nova-invoice-viewer` | `Viewer` (template `NOVAInvoiceSuiteViewer`) | tutti gli utenti read-only | | `nova-invoice-worker` | `Worker` | operativi day-to-day | | `nova-invoice-approver` | `Approver` | approvatori standard | | `nova-invoice-approver-manager` | `Approver`, `ApproverManager` | manager con override BP/PO | | `nova-invoice-approver-tax` | `Approver`, `ApproverTax` | controller fiscale | | `nova-invoice-approver-senior` | `Approver`, `ApproverSenior` | direttore finanziario | | `nova-invoice-approver-director` | `Approver`, `ApproverDirector` | direttore generale | | `nova-invoice-approver-cfo` | `Approver`, `ApproverCFO` | CFO (top tier) | | `nova-invoice-posting-officer` | `PostingOfficer` | contabilitΓ  (SoD mutex con Approver) | | `nova-invoice-admin` | `Admin` | amministratore CC | | `nova-invoice-admin-config` | `AdminConfig` | sviluppatori/consulenti config-only | | `nova-invoice-operational-admin` | `OperationalAdmin` | ops day-to-day (esente SoD) | | `nova-invoice-flex-key-user` | `FlexKeyUser` | personalization variants | | `nova-invoice-super-admin` | `SuperAdmin` | sysadmin globale (audit-logged) | > **CompanyCode attribute mapping** (richiesto): in ogni Role Collection, `Edit β†’ User Attributes β†’ CompanyCode`. Source: `IDP / IAS group β†’ 'CompanyCode' attribute`. #### Esempio: Role Collection `nova-invoice-worker` ``` Name: nova-invoice-worker Description: Worker - operational invoice processing Roles: - NOVAInvoiceSuiteWorker (from XSUAA) User Groups: (assign group-by-group via IAS β€” vedi Β§5) User Attributes: - CompanyCode: from IAS group attribute ``` ### 4.6 Trust subaccount β†’ IAS 1. Subaccount β†’ **Security β†’ Trust Configuration** 2. **Establish Trust** β†’ seleziona IAS tenant configurato a Global Account (Β§3.3) 3. Default ID provider: `IAS` 4. Email: `Email` (mapping standard) 5. Save ## 5. SAP IAS β€” configurazione applicazione + groups > **Da fare una volta per IAS tenant. Configurazione completa Γ¨ in [IAS_GROUPS_SETUP.md](IAS_GROUPS_SETUP.md).** ### 5.1 Crea applicazione SAML in IAS 1. Login `https://.accounts.ondemand.com/admin` 2. **Applications & Resources β†’ Applications β†’ Create** 3. Name: `NOVA Invoice Suite β€” production` 4. Type: `OpenID Connect / OAuth 2.0` 5. **Save** ### 5.2 OIDC configuration Click sull'application appena creata β†’ **Trust β†’ SAML 2.0 / OIDC Configuration**: ``` Subject Name Identifier: Email Default Name ID Format: Email Assertion Attributes: - mail (default) - first_name (default) - last_name (default) - groups (custom β€” vedi Β§5.3) - CompanyCode (custom) ``` Annota: - **Issuer**: `https://.accounts.ondemand.com` - **Audience / Client ID**: visibile in tab `Client Authentication` (genera un Secret se manca) > Userai questi valori in env `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_AUDIENCE`. ### 5.3 Custom assertion attributes Per multi-tenant CompanyCode mapping: 1. **Application β†’ Trust β†’ SAML 2.0 / OIDC Configuration β†’ Assertion Attributes** 2. **Add Custom Attribute**: - Name: `CompanyCode` - Source: `User Store` - User Attribute: scegli campo (es. `customAttribute1`) o sincronizzato da Active Directory 3. **Add Custom Attribute**: - Name: `groups` - Source: `Groups` (built-in) ### 5.4 IAS Groups In IAS: 1. **User Management β†’ Groups β†’ Create** 2. Per ogni ruolo applicativo crea un gruppo: ``` nova.invoice.Viewer nova.invoice.Worker nova.invoice.Approver nova.invoice.ApproverManager nova.invoice.ApproverTax nova.invoice.ApproverSenior nova.invoice.ApproverDirector nova.invoice.ApproverCFO nova.invoice.PostingOfficer nova.invoice.Admin nova.invoice.AdminConfig nova.invoice.OperationalAdmin nova.invoice.FlexKeyUser nova.invoice.SuperAdmin ``` 3. Assegna utenti ai gruppi (manualmente o via SCIM provisioning) ### 5.5 Group β†’ Role Collection mapping In **BTP Subaccount β†’ Trust Configuration β†’ IAS β†’ Default Role Collection Mapping**: ``` nova.invoice.Viewer β†’ nova-invoice-viewer nova.invoice.Worker β†’ nova-invoice-worker nova.invoice.Approver β†’ nova-invoice-approver nova.invoice.ApproverManager β†’ nova-invoice-approver-manager ... (etc per tutti i 14 + system-user) ``` User attribute mapping: ``` CompanyCode (Role Collection) ← CompanyCode (IAS assertion) ``` ### 5.6 Conditional Authentication (opzionale) Per MFA su SuperAdmin: 1. **Application β†’ Authentication and Access β†’ Conditional Authentication** 2. Add Rule: - Conditions: `groups CONTAINS 'nova.invoice.SuperAdmin'` - Authentication Method: `Two-Factor Authentication (TOTP)` ### 5.7 SCIM API per user provisioning automatizzato Se hai HR system (SuccessFactors, Active Directory, ecc.): 1. IAS β†’ **Application β†’ Provisioning β†’ SCIM API** 2. Generate OAuth client + secret 3. Configure HR system to push users + group membership via SCIM ## 6. BTP CF deployment β€” step-by-step ### 6.1 Login + target ```bash cf login -a https://api.cf..hana.ondemand.com \ -u -o -org -s production --sso ``` ### 6.2 Crea service instances ```bash # XSUAA application cf create-service xsuaa application nova-uaa -c xs-security.json # HANA HDI container cf create-service hana hdi-shared nova-db # Destination service cf create-service destination lite nova-destination # HTML5 Application Repository cf create-service html5-apps-repo app-host nova-html5-host cf create-service html5-apps-repo app-runtime nova-html5-runtime # Audit Log cf create-service auditlog premium nova-auditlog # Cloud Logging cf create-service application-logs standard nova-logging # Alert Notification cf create-service alert-notification standard nova-alerts # Job Scheduling cf create-service jobscheduler standard nova-jobs # Enterprise Messaging cf create-service enterprise-messaging default-cf nova-messaging \ -c '{"emname":"nova-em","namespace":"nova/invoice/v1"}' # DOX cf create-service document-information-extraction default nova-dox # Redis cf create-service redis-cache development nova-redis # Autoscaler cf create-service autoscaler standard nova-autoscaler # CIAS cf create-service cloud-integration-automation standard nova-cias # AI Core (opzionale) cf create-service aicore extended nova-aicore # Workzone (opzionale per Launchpad) cf create-service workzone-standard standard nova-workzone # UI5 Flex (opzionale) cf create-service sap-ui5-flexibility-keyuser standard nova-ui5-flex # Responsibility Management (opzionale) cf create-service responsibility-mgmt standard nova-respmgmt ``` Wait for all to be `create succeeded`: ```bash cf services ``` ### 6.3 Configure destinations In BTP Cockpit β†’ **Subaccount β†’ Connectivity β†’ Destinations**: #### S/4HANA destinations (uno per API) | Name | URL | Authentication | |---|---|---| | `S4_BUSINESS_PARTNER` | `https://.s4hana.cloud.sap` | OAuth2ClientCredentials | | `S4_PURCHASE_ORDER` | id. | id. | | `S4_SERVICE_ENTRY_SHEET` | id. | id. | | `S4_OPLACCTGDOCITEMCUBE` | id. | id. | | `S4_SUPPLIER_INVOICE` | `https://-api.s4hana.cloud.sap` | id. | | `S4_ELECTRONIC_DOC_FILE` | id. | id. | Per ognuna, additional properties: ``` WebIDEEnabled=true HTML5.DynamicDestination=true HTML5.Timeout=30000 sap-client= # se necessario ``` #### IAS destination ``` Name: NOVA-IAS URL: https://.accounts.ondemand.com Authentication: OAuth2SAMLBearerAssertion TokenServiceURL: https://.accounts.ondemand.com/oauth2/token TokenServiceUser: TokenServicePassword: audience: nova-invoice-suite ``` ### 6.4 Build MTA ```bash mbt build -p=cf # output: mta_archives/NOVAInvoiceSuite_.mtar ``` Build time: ~3-5 min (per build piΓΉ veloce: `mbt build -p=cf --mtar nova.mtar`). ### 6.5 Deploy ```bash cf deploy mta_archives/NOVAInvoiceSuite_*.mtar ``` Deploy time: ~10-15 min (initial) / ~3-5 min (delta updates). Watch progress: ```bash cf logs nova-srv --recent cf apps ``` Expected modules: ``` nova-srv (1024M, 1 instance) nova-approuter (512M, 1 instance) nova-db-deployer (256M, completed) nova-html5-host (HTML5 module β€” no compute) ``` ### 6.6 Post-deploy: assign users 1. Subaccount β†’ **Security β†’ Users β†’ Add User** 2. Origin: `` (nuovo IdP, NON `sap.default`) 3. Email 4. Assign role collection (es. `nova-invoice-admin`) ### 6.7 Custom domain (opzionale) ```bash # Map custom domain cf map-route nova-approuter nova.example.com # Verify SSL curl -I https://nova.example.com ``` ### 6.8 BTP CF β€” verifica end-to-end ```bash # Health curl https://nova-approuter--production.cfapps..hana.ondemand.com/health # Service catalog open https://nova-approuter--production.cfapps..hana.ondemand.com/ # Login con utente assegnato ``` ## 7. BTP Kyma deployment β€” step-by-step ### 7.1 Pre-flight ```bash # Download kubeconfig dalla BTP Cockpit mv ~/Downloads/kubeconfig.yaml ~/.kube/nova-kyma-kubeconfig export KUBECONFIG=~/.kube/nova-kyma-kubeconfig # Verify kubectl cluster-info kubectl get nodes # should show 3 nodes Ready ``` ### 7.2 Crea namespace + service account CI/CD > **Manual step nel Kyma Dashboard**: > > 1. Login Kyma Dashboard via BTP Cockpit > 2. **Namespaces β†’ Create Namespace** β†’ name: `nova-invoice-suite`, enable Istio injection > 3. **Configuration β†’ Service Accounts β†’ Create**: > - Name: `nova-ci-deployer` > - Namespace: `nova-invoice-suite` > 4. **Roles & Bindings β†’ Create Role**: > - Name: `nova-ci-deployer-role` > - Permissions: `apps/*`, `core/*`, `batch/*`, `gateway.kyma-project.io/*` (full) > 5. **Roles & Bindings β†’ Create RoleBinding** (YAML view): > ```yaml > apiVersion: rbac.authorization.k8s.io/v1 > kind: RoleBinding > metadata: > name: nova-ci-deployer-binding > namespace: nova-invoice-suite > subjects: > - kind: ServiceAccount > name: nova-ci-deployer > namespace: nova-invoice-suite > roleRef: > kind: Role > name: nova-ci-deployer-role > apiGroup: rbac.authorization.k8s.io > ``` > 6. **Secrets β†’ Create** (YAML view): > ```yaml > apiVersion: v1 > kind: Secret > type: kubernetes.io/service-account-token > metadata: > name: nova-ci-deployer-token > namespace: nova-invoice-suite > annotations: > kubernetes.io/service-account.name: nova-ci-deployer > ``` > 7. Wait 10s, retrieve token via `kubectl get secret nova-ci-deployer-token -o jsonpath='{.data.token}' | base64 -d` ### 7.3 Configure secrets #### `nova-oidc-credentials` (IAS) ```bash kubectl create secret generic nova-oidc-credentials \ --from-literal=issuer="https://.accounts.ondemand.com" \ --from-literal=client-id="" \ --from-literal=audience="nova-invoice-suite" \ -n nova-invoice-suite ``` #### `nova-job-token` ```bash kubectl create secret generic nova-job-token \ --from-literal=token="$(openssl rand -hex 32)" \ -n nova-invoice-suite ``` #### `nova-billing-token` (separato per security) ```bash kubectl create secret generic nova-billing-token \ --from-literal=token="$(openssl rand -hex 32)" \ -n nova-invoice-suite ``` #### `nova-mcp-token` ```bash kubectl create secret generic nova-mcp-token \ --from-literal=token="$(openssl rand -hex 32)" \ --from-literal=allowed-cc="1000,2000,3000" \ -n nova-invoice-suite ``` #### `nova-s4-credentials` ```bash kubectl create secret generic nova-s4-credentials \ --from-literal=base-url="https://.s4hana.cloud.sap" \ --from-literal=api-base-url="https://-api.s4hana.cloud.sap" \ --from-literal=client-id="" \ --from-literal=client-secret="" \ -n nova-invoice-suite ``` #### `nova-s3-credentials` (DMS via MinIO) Se usi MinIO esterno: ```bash kubectl create secret generic nova-s3-credentials \ --from-literal=endpoint="https://minio.example.com" \ --from-literal=bucket="nova-invoice-archive" \ --from-literal=access-key="" \ --from-literal=secret-key="" \ --from-literal=region="us-east-1" \ -n nova-invoice-suite ``` ### 7.4 Deploy PostgreSQL in-cluster (se non gestito esternamente) ```bash helm repo add bitnami https://charts.bitnami.com/bitnami helm install nova-postgres bitnami/postgresql \ -n nova-invoice-suite \ --set auth.postgresPassword="$(openssl rand -hex 16)" \ --set auth.database=nova \ --set primary.persistence.size=10Gi \ --set primary.resources.requests.memory=512Mi ``` Get connection string: ```bash PG_PWD=$(kubectl get secret nova-postgres-postgresql -n nova-invoice-suite -o jsonpath='{.data.postgres-password}' | base64 -d) echo "postgres://postgres:${PG_PWD}@nova-postgres-postgresql.nova-invoice-suite:5432/nova" ``` ### 7.5 ConfigMap ```bash kubectl apply -f k8s/configmap.yaml ``` `configmap.yaml`: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: nova-config namespace: nova-invoice-suite data: CDS_ENV: "k8s" NODE_ENV: "production" CDS_TYPESCRIPT: "true" LOG_LEVEL: "info" CUSTOMER_ID: "${CUSTOMER_ID}" KYMA_APP_HOST: "nova-${CUSTOMER_ID}.kyma.ondemand.com" ``` ### 7.6 PVC (per filesystem DMS opzionale) ```bash kubectl apply -f k8s/pvc.yaml ``` ### 7.7 Service Bindings BTP managed services (opzionale) Per usare BTP DMS / Audit Log da Kyma: ```bash # DOX binding cat <.kyma.ondemand.com` | APIRule hostname | | `OIDC_ISSUER` | IAS tenant URL | IAS app config | | `OIDC_CLIENT_ID` | IAS app client ID | id. | | `OIDC_AUDIENCE` | `nova-invoice-suite` | IAS app config | | `S4_BASE_URL` | `https://.s4hana.cloud.sap` | | | `S4_API_BASE_URL` | `https://-api.s4hana.cloud.sap` | | | `S4_CLIENT_ID` | OAuth2 client ID | S/4 admin | | `S4_CLIENT_SECRET` | OAuth2 secret | id. | | `GHCR_TOKEN` | GitHub PAT con `write:packages` | per push image | ### 7.9 Trigger deploy ```bash gh workflow run deploy.yml --ref feature/lifecycle-drift-r1 # o push to main triggera automaticamente ``` Watch: ```bash gh run watch ``` ### 7.10 Manual deploy (alternative) ```bash # Build + push image docker build -t ghcr.io//nova-srv:$(git rev-parse --short HEAD) -f Dockerfile . docker push ghcr.io//nova-srv:$(git rev-parse --short HEAD) # Apply manifests export IMAGE_TAG=$(git rev-parse --short HEAD) export KYMA_APP_HOST=nova-.kyma.ondemand.com export OIDC_ISSUER=https://.accounts.ondemand.com envsubst < k8s/deployment.yaml | kubectl apply -f - envsubst < k8s/apirule.yaml | kubectl apply -f - kubectl apply -f k8s/service.yaml kubectl apply -f k8s/cronjob-auto-advance.yaml # ... altri cronjob ``` ### 7.11 Verify ```bash # Pods kubectl get pods -n nova-invoice-suite # Expected: nova-srv-* Running, nova-approuter-* Running, nova-postgres-* Running # APIRule kubectl get apirules -n nova-invoice-suite # Health curl https://${KYMA_APP_HOST}/health # Logs kubectl logs -n nova-invoice-suite -l app=nova-srv -f ``` ## 8. On-premise (k3d/k3s) deployment Per air-gapped o sovereignty data: ### 8.1 Setup k3d cluster ```bash brew install k3d k3d cluster create nova-onprem \ --servers 1 --agents 2 \ --port "443:443@loadbalancer" \ --port "80:80@loadbalancer" ``` ### 8.2 Install Keycloak ```bash helm repo add bitnami https://charts.bitnami.com/bitnami helm install nova-keycloak bitnami/keycloak \ -n nova-invoice-suite --create-namespace \ --set auth.adminUser=admin \ --set auth.adminPassword="$(openssl rand -hex 16)" ``` Realm import: ```bash kubectl cp k8s/onprem/keycloak-realm.json nova-keycloak-0:/tmp/realm.json -n nova-invoice-suite kubectl exec nova-keycloak-0 -n nova-invoice-suite -- /opt/bitnami/keycloak/bin/kc.sh import --file /tmp/realm.json ``` ### 8.3 Install MinIO ```bash helm install nova-minio bitnami/minio \ -n nova-invoice-suite \ --set auth.rootUser=admin \ --set auth.rootPassword="$(openssl rand -hex 16)" \ --set defaultBuckets="nova-invoice-archive" ``` ### 8.4 Install Loki + Prometheus + Grafana ```bash helm install nova-loki grafana/loki-stack \ -n observability --create-namespace \ --set grafana.enabled=true \ --set prometheus.enabled=true ``` ### 8.5 Deploy NOVA con profile k8s-onprem ```bash export KUBECONFIG=$(k3d kubeconfig write nova-onprem) export CDS_ENV=k8s-onprem # Procedi come in Β§7.5+ ``` Vedi [k8s/onprem/tls-setup.md](../k8s/onprem/tls-setup.md) per TLS. ## 9. Post-deploy verification ### 9.1 Smoke test endpoints ```bash # Health (no auth) curl https://${APP_HOST}/health # OData metadata (richiede JWT) curl -H "Authorization: Bearer $JWT" \ https://${APP_HOST}/odata/v4/orchestrator/\$metadata ``` ### 9.2 Run lifecycle E2E in production-like ```bash # Locale, contro deploy export NOVA_BASE_URL=https://${APP_HOST} npm run test:e2e ``` ### 9.3 Validate role collections Crea un utente test con ogni role collection, verifica che le button corrispondenti siano abilitate/disabilitate. ### 9.4 Cron jobs ```bash # Kyma kubectl get cronjobs -n nova-invoice-suite # CF cf service nova-jobs # vedi configured jobs ``` ### 9.5 SystemParameters caricati Dopo `cds deploy`: ```sql SELECT COUNT(*) FROM sap_passive_invoice_SystemParameters; -- expected: 100+ rows da CSV seed ``` ## 10. Update / rollback ### Update ```bash # CF mbt build -p=cf cf deploy mta_archives/NOVAInvoiceSuite_.mtar -f # Kyma gh workflow run deploy.yml --ref main # o manual: rebuild image + kubectl apply ``` ### Rollback #### CF β€” bg-deploy strategy ```bash # bg-deploy crea nuova versione live, mantiene old per rollback cf bg-deploy mta_archives/NOVAInvoiceSuite_.mtar # Rollback se serve cf undeploy NOVAInvoiceSuite --abort # solo se ancora in deploy cf rollback # versione precedente ``` #### Kyma ```bash # Rollback Deployment kubectl rollout undo deployment/nova-srv -n nova-invoice-suite # Verify kubectl rollout status deployment/nova-srv -n nova-invoice-suite ``` --- ## 10-bis. GitHub Actions CI/CD pipelines NOVA ha 7 workflow attivi in [.github/workflows/](../.github/workflows/) β€” uno per ogni scenario operativo. ### 10-bis.1 Inventario workflow | Workflow | Trigger | Scope | Stato | |---|---|---|---| | **deploy.yml** | Push to `main`, manual dispatch | Build Docker + push GHCR + `kubectl apply` Kyma | βœ… Primary CI/CD | | **_deploy-customer.yml** | Reusable workflow (called by deploy.yml + manual) | Single-customer deploy con `dr_tier` input + `customer_name` envsubst | βœ… Multi-customer fan-out | | **billing-monthly.yml** | Cron `0 2 1 * *` (1st of month, 02:00 UTC) | Mensile billing aggregation via `scripts/billing-aggregate.js` | βœ… Production | | **deploy-minio.yml** | Manual dispatch + push to `k8s/storage/**` | Infrastructure: deploy MinIO standalone (S3-compatible storage) | βœ… Optional, `dms.adapter=s3` | | **pg-nightly.yml** | Cron `0 3 * * *` (03:00 UTC) | Smoke test profilo `production-pg` (DEPRECATED) | 🟑 Nightly (verify se ancora attivo) | | **security.yml** | Push to `main`, PR | ZAP scan + dependency audit + SAST | βœ… Production gate | | **deploy-piper.yml** | Manual dispatch (gated by var) | Phase B cTMS integration via SAP Piper steps | ⚠️ DORMANT (template per future cTMS opt-in) | ### 10-bis.2 deploy.yml β€” Pipeline CI/CD primaria **Path**: [.github/workflows/deploy.yml](../.github/workflows/deploy.yml) **Steps**: 1. **Test**: `npm test` (1381 unit) + `npm run lint` + `npm run typecheck:strict` 2. **Build Docker**: `docker build -f Dockerfile -t ghcr.io//nova-invoice-suite-srv:sha-` 3. **Push GHCR**: `docker push ghcr.io//nova-invoice-suite-srv:sha-` (image pubblica, evita 401 imagepull) 4. **Customer fan-out**: per ogni customer, chiama `_deploy-customer.yml` con `customer_name` + `dr_tier` 5. **kubectl apply**: envsubst su `k8s/*.yaml` + `kubectl apply` (no Helm) **Secrets richiesti** (GitHub Settings β†’ Secrets): - `GHCR_TOKEN` β€” GitHub PAT con `write:packages` - `KYMA_KUBECONFIG_` β€” base64 kubeconfig per cluster - Vedi [DEPLOYMENT_GUIDE Β§7.5](#75-github-secrets-environment-production) per lista completa **Bypass via `[skip ci]`**: - Commit message contenente `[skip ci]` skip-pa il workflow - Usato per docs-only changes (es. commit `docs(overhaul)` 284f569) ### 10-bis.3 _deploy-customer.yml β€” Reusable single-customer **Path**: [.github/workflows/_deploy-customer.yml](../.github/workflows/_deploy-customer.yml) **Inputs**: ```yaml customer_name: { required: true, type: string } dr_tier: { required: false, default: "T0_SINGLE_REGION", type: string } image_tag: { required: true, type: string } ``` **Logic**: 1. Resolve customer-specific manifests via [k8s/overlays//](../k8s/overlays/) (Kustomize) 2. Apply DR tier overlay se `dr_tier != T0_SINGLE_REGION` 3. envsubst su `IMAGE_TAG`, `CUSTOMER_ID`, `KYMA_APP_HOST`, `OIDC_ISSUER` 4. `kubectl apply -f -` con kubeconfig customer 5. Post-deploy smoke test su `/health` ### 10-bis.4 deploy-minio.yml β€” MinIO storage infrastructure **Path**: [.github/workflows/deploy-minio.yml](../.github/workflows/deploy-minio.yml) **Quando usarlo**: customer abilita `DMS_ADAPTER=s3` (vedi [ADR 0002](adr/0002-archivelink-cs-storage-backing.md)). **Inputs**: ```yaml action: { required: true, choices: ["apply", "init-buckets", "status"] } ``` **Trigger**: - `workflow_dispatch` (manual) β€” recommended per first deploy - Push a `k8s/storage/**` (auto on manifest changes) **Environment**: `production` con required reviewers + branch restriction `main`. **Secrets**: - `MINIO_ACCESS_KEY` (environment-level, NOT repo-level) - `MINIO_SECRET_KEY` (environment-level) **Use case**: 1. **First deploy**: `action=apply` β†’ crea PVC + StatefulSet MinIO + Service 2. **Bucket init**: `action=init-buckets` β†’ crea bucket `nova-archivelink`, `nova-conservatore`, `nova-pg-backup` 3. **Status check**: `action=status` β†’ verifica pod ready + PVC bound ### 10-bis.5 security.yml β€” Security gate **Path**: [.github/workflows/security.yml](../.github/workflows/security.yml) **Steps**: 1. **OWASP ZAP scan** baseline contro `https://${{ secrets.ZAP_TARGET }}` (URL definito a livello secret per region-aware) 2. **npm audit** β€” fail su `--audit-level=high` 3. **SAST**: GitHub CodeQL su `srv/` + `app/` 4. **Dependency review** (PR only): blocca PR se introduce CVE high **Run frequency**: - Push to `main`: full scan - PR: dependency review + npm audit (no ZAP per costo) - Schedule: weekly full ZAP scan (cron) **Failure handling**: blocca merge PR fino a fix (gate quality). ### 10-bis.6 pg-nightly.yml β€” PostgreSQL profile smoke **Path**: [.github/workflows/pg-nightly.yml](../.github/workflows/pg-nightly.yml) **Status**: 🟑 Verifica trimestrale β€” se non eseguito >30gg, demote `production-pg` a "untested" in CLAUDE.md. **Steps**: 1. Spin up PostgreSQL container in CI 2. `npm run start:backend -- --profile production-pg` 3. Smoke test 50 invoices end-to-end 4. Tear down **Use case**: validate `mta-pg.yaml` (DEPRECATED ma ancora supported fino a 2026-Q4 retirement). ### 10-bis.7 deploy-piper.yml β€” DORMANT **Path**: [.github/workflows/deploy-piper.yml](../.github/workflows/deploy-piper.yml) **Status**: ⚠️ DORMANT β€” non triggera automaticamente. **Activation**: setta `vars.TRANSPORT_CODE_ADAPTER == 'github_actions_piper'` (default `github_actions`). **Use case future**: customer enterprise con SAP Piper governance + cTMS native deployment. Vedi [docs/ctms-integration.md](ctms-integration.md) Β§"MODE 4 Piper". ### 10-bis.8 Best practice CI/CD | Practice | Why | |---|---| | **Image SHA pinning** | `image: ghcr.io/...@sha256:` invece di tag mutable | | **Required reviewers** su `production` env | Manual approval per deploy critici (deploy-minio.yml) | | **Branch protection** `main`: 2 reviewer + status checks pass | Evita merge senza CI green | | **Secrets ruotati quarterly** | Vedi [TOKEN_ROTATION.md](runbooks/TOKEN_ROTATION.md) | | **`[skip ci]` solo per docs-only** | Mai bypassare CI per code changes | | **Customer-scoped state**: `kubeconfig` per customer in env-level secret | Evita cross-customer accidental deploy | ### 10-bis.9 Code transport modes β€” cTMS + Piper NOVA Invoice Suite supporta **4 mode di code transport** alternative a (o coesistenti con) la pipeline Kyma Docker primaria. La selezione si fa via GitHub variable `TRANSPORT_CODE_ADAPTER` (default `github_actions`). Per la documentazione completa vedi [docs/ctms-integration.md](ctms-integration.md). Sintesi rapida: | Mode | `TRANSPORT_CODE_ADAPTER` | Runner | Step library | Deploy target | Activation | |---|---|---|---|---|---| | **1** (default) | `github_actions` | GitHub-hosted | custom bash + `kubectl` | Kyma (Docker) | sempre attiva | | **2** | `github_actions_ctms` | GitHub-hosted | custom bash + curl TMS REST | cTMS (MTA) | job dormant in `deploy.yml` | | **3** | `sap_btp_cicd` | **SAP BTP managed** | **Project Piper (managed)** | cTMS (nativo) | `.pipeline/config.yml` registrato in BTP CI/CD UI | | **4** | `github_actions_piper` | GitHub-hosted | **Piper via `SAP/project-piper-action`** | cTMS / CF / Kyma | workflow dormant `deploy-piper.yml` | **Quando scegliere quale**: - **MODE 1** = customer Kyma-only, no governance cTMS (default). - **MODE 2** = customer ha cTMS ma vuole tenere visibilitΓ  GitHub (script self-rolled curl). - **MODE 3** = customer enterprise con audit SOX, vuole runner managed SAP + Cloud ALM nativo + zero-secret service binding. - **MODE 4** = customer GHA-first ma vuole step Piper ufficiali SAP-maintained (Checkmarx scan, mtaBuild standard, tmsExport). **Coesistenza**: MODE 1 + MODE 3 Γ¨ valido (Kyma + cTMS dual deploy su artifact diversi). MODE 2/3/4 si escludono per il codice transport. #### 10-bis.9.1 Activation steps per MODE 2 (`github_actions_ctms`) **Pre-requisiti**: - BTP entitlement `cloud-transport-management` plan `export` (vedi Β§4.2). - Service instance `nova-tms` + service key (estrarre `client_id`/`client_secret`/`endpoints.token`/`uri`). - TMS landscape configurato (nodi `NOVA_CODE_DEV` β†’ `NOVA_CODE_QA` β†’ `NOVA_CODE_PROD` + routes). **GitHub repo settings**: 1. Variables β†’ `TRANSPORT_CODE_ADAPTER = github_actions_ctms`, opzionalmente `CTMS_SOURCE_NODE_CODE = NOVA_CODE_DEV`. 2. Secrets β†’ `TMS_OAUTH_CLIENT_ID`, `TMS_OAUTH_CLIENT_SECRET`, `TMS_OAUTH_TOKEN_URL`, `TMS_URI`. 3. Push su `main` β†’ job `ctms-export-mta` in `deploy.yml` esegue MBT build + curl POST upload + create export request. #### 10-bis.9.2 Activation steps per MODE 3 (`sap_btp_cicd`) **Pre-requisiti**: - BTP subscription `cicd-service` plan `standard` (entitlement + subaccount-level). - Service instance + service key per `cloud-transport-management`. - BTP CI/CD Service UI access. **Setup** (vedi `.pipeline/README.md` per step dettagliati): 1. BTP Cockpit β†’ Services β†’ CI/CD Service β†’ Open dashboard. 2. Repositories β†’ Add β†’ URL GitHub repo + credentials (PAT o app GitHub). 3. Jobs β†’ Add β†’ name `nova-invoice-suite`, branch `main`, pipeline source `.pipeline/config.yml`. 4. Webhook GitHub configurato automaticamente (o manuale via repo Settings β†’ Webhooks). 5. Service bindings: `nova-tms-cicd` (cloud-transport-management) + opzionale `nova-alert-notification` (alert-notification). 6. SystemParameter runtime β†’ `TRANSPORT_CODE_ADAPTER = sap_btp_cicd`. 7. Push su `main` β†’ BTP CI/CD esegue Build/Test/Release/Notify stages (runner managed da SAP). #### 10-bis.9.3 Activation steps per MODE 4 (`github_actions_piper`) **Pre-requisiti**: come MODE 2 + Piper action gironegerito da SAP. **GitHub repo settings**: 1. Variables β†’ `TRANSPORT_CODE_ADAPTER = github_actions_piper`, opzionalmente `CTMS_SOURCE_NODE_CODE`, `PIPER_SECURITY_SCANS=true`. 2. Secrets β†’ `PIPER_TMS_SERVICE_KEY` (full service-key JSON come singola stringa), opzionalmente `CHECKMARX_USERNAME`/`CHECKMARX_PASSWORD`, `CF_USERNAME`/`CF_PASSWORD`. 3. Push su `main` β†’ workflow `deploy-piper.yml` triggera step `piper-build` β†’ `piper-scan` (opzionale) β†’ `piper-tms-export`. **Versione Piper pinnata**: `SAP/project-piper-action@v1.27.1` β€” vedi header del workflow per checklist upgrade. #### 10-bis.9.4 Customizing transport runtime (TRANSPORT_CUSTOMIZING_ADAPTER) Indipendente dalla code transport, il **customizing transport** (export/import config bundle ApprovalRule, ServiceConnector, SystemParameters, ecc.) usa 2 adapter selezionabili via `SystemParameters.INFRASTRUCTURE.TRANSPORT_CUSTOMIZING_ADAPTER`: | Valore | Adapter | Implementazione | |---|---|---| | `manual` (default) | `ManualTransportAdapter` | JSON download/upload via UI Setup app `Configurazione Sistema` Β§ Trasporto | | `ctms` (opt-in) | `CTMSTransportAdapter` | Application Content `.zip` (`manifest.json` + `bundle.json`) + cTMS `/v2/files/upload` + `/v2/nodes/export` | **Fail-safe**: se `ctms` selected ma destination/binding non disponibili β†’ factory downgrada silentemente a `manual` con WARN log. CAP parte sempre. **Riferimenti codice**: `srv/transport/{TransportAdapter,Manual,CTMS}TransportAdapter.ts`, `srv/transport/TransportAdapterFactory.ts`. --- ## 10-ter. Observability β€” SAP Cloud Logging NOVA Invoice Suite invia log applicativi, metriche e tracce a **SAP Cloud Logging** (servizio observability OpenSearch-based su BTP). Setup multi-runtime (CF + Kyma) con 4 path di provisioning supportati ufficialmente. Riferimento: [SAP Help β€” What is Cloud Logging](https://help.sap.com/docs/btp/sap-business-technology-platform/what-is-sap-cloud-logging). ### 10-ter.1 Architettura ingestion ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” Pod CAP (nova-srv) ──── stdout JSON ──┐ β”‚ SAP Cloud Logging β”‚ β”œβ”€β”€β”€β”€ OTLP ─→ (OpenSearch + UI) β”‚ @cap-js/telemetry ─── traces/metrics β”€β”€β”˜ β”‚ β€’ logs-json-kyma* β”‚ β”‚ β€’ OpenTelemetry β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ indices β”‚ β”‚ Kyma Telemetry CR β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β€’ LogPipeline (FB) β”‚ ──────── push ────────────────▲ β”‚ β€’ TracePipeline (OTLP) β”‚ β”‚ mTLS client cert β”‚ β€’ MetricPipeline β”‚ β”‚ from binding secret β”‚ β€’ Istio (envoy) β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ CF firehose ─────────── stdout capture ─────────── VCAP_SERVICES OTLP endpoint ``` Per dettagli architetturali completi (PII redaction, correlation IDs, semantic conventions): vedi [`docs/explanation/06-logging-architecture.md`](explanation/06-logging-architecture.md). ### 10-ter.2 Service plan & retention | Plan | Retention max | Index volume | Use case | |---|---|---|---| | `dev` | 7 giorni | small | dev/pilot | | `standard` (default progetto) | 90 giorni | medium | production single-tenant | | `large` | 90 giorni | high | enterprise multi-tenant | Configurazione attuale: `standard` plan, **30 giorni** retention (compromise GDPR data minimization vs. troubleshooting window). Vedi [`mta.yaml:355-386`](../mta.yaml). ### 10-ter.3 Provisioning β€” 4 path supportati #### Option A β€” BTP Cockpit (manuale, raccomandato per primo setup) 1. BTP Cockpit β†’ Subaccount β†’ Services β†’ Service Marketplace β†’ cerca **"Cloud Logging"**. 2. **Create instance** β†’ seleziona plan (`standard`), region. 3. **Parameters** (configurazione iniziale, modificabile post-creation): ```json { "retention_period": 30, "ingest_otlp": { "enabled": true }, "saml": { "enabled": true, "idp": { "metadata_url": "https://.accounts.ondemand.com/saml2/metadata", "entity_id": "" }, "sp": { "entity_id": "cloud-logging-nova-invoice-suite" }, "admin_group": "NOVA-CloudLogging-Admin", "roles_key": "groups" } } ``` 4. **Create** β†’ instance provisioning ~5min. 5. **Create Service Key** β†’ name `nova-cls-binding` β†’ estrai endpoints + cert per binding. β†’ Doc ufficiale: [Create through BTP Cockpit](https://help.sap.com/docs/btp/sap-business-technology-platform/create-an-sap-cloud-logging-instance-through-sap-btp-cockpit). #### Option B β€” Cloud Foundry CLI ```bash cf login -a cf create-service cloud-logging standard NOVAInvoiceSuite-cloud-logging \ -c '{ "retention_period": 30, "ingest_otlp": { "enabled": true } }' # Verify async creation (puΓ² richiedere 2-5 min) cf service NOVAInvoiceSuite-cloud-logging # Service key per export endpoint + cert cf create-service-key NOVAInvoiceSuite-cloud-logging nova-cls-binding cf service-key NOVAInvoiceSuite-cloud-logging nova-cls-binding | jq . ``` β†’ Doc ufficiale: [Create through CF CLI](https://help.sap.com/docs/btp/sap-business-technology-platform/create-an-sap-cloud-logging-instance-through-cloud-foundry-cli). #### Option C β€” BTP CLI (preferred per CI/CD provisioning script) ```bash btp login --url https://cli.btp.cloud.sap --user admin@... --subdomain btp target --subaccount btp create services/instance \ --offering-name cloud-logging \ --plan-name standard \ --name NOVAInvoiceSuite-cloud-logging \ --parameters '{ "retention_period": 30, "ingest_otlp": { "enabled": true } }' btp create services/binding \ --name nova-cls-binding \ --instance-name NOVAInvoiceSuite-cloud-logging ``` β†’ Doc ufficiale: [Create through BTP CLI](https://help.sap.com/docs/btp/sap-business-technology-platform/create-an-sap-cloud-logging-instance-through-sap-btp-cli). #### Option D β€” BTP Service Operator (Kyma β€” utilizzato dal progetto) Il progetto usa BTP Service Operator per Kyma. Manifest in [`k8s/telemetry/cls-instance.yaml`](../k8s/telemetry/cls-instance.yaml): ```yaml apiVersion: services.cloud.sap.com/v1 kind: ServiceInstance metadata: name: nova-cloud-logging namespace: nova-invoice-suite spec: serviceOfferingName: cloud-logging servicePlanName: standard parameters: retention_period: 30 ingest_otlp: enabled: true --- apiVersion: services.cloud.sap.com/v1 kind: ServiceBinding metadata: name: nova-cls-binding namespace: nova-invoice-suite spec: serviceInstanceName: nova-cloud-logging secretName: nova-cls-binding ``` Apply: `kubectl apply -f k8s/telemetry/cls-instance.yaml`. Service Operator crea instance + secret `nova-cls-binding` automaticamente (5-10min provisioning). β†’ Doc ufficiale: [Create through Service Operator](https://help.sap.com/docs/btp/sap-business-technology-platform/create-an-sap-cloud-logging-instance-through-sap-btp-service-operator). #### Option E β€” Terraform (IaC) ```hcl # terraform/main.tf resource "btp_subaccount_entitlement" "cloud_logging" { count = var.enable_cloud_logging ? 1 : 0 subaccount_id = local.subaccount_id service_name = "cloud-logging" plan_name = "standard" } ``` Activation: `enable_cloud_logging = true` in `clients/.tfvars`. **Service instance provisioning** richiede comunque Option B/C/D post-entitlement (Terraform provisiona solo entitlement nel global account). ### 10-ter.4 Ingestion setup Kyma 3 Telemetry CR in [`k8s/telemetry/`](../k8s/telemetry/) (apply post-Service Operator): | File | Risorsa | Scopo | |---|---|---| | `log-pipeline.yaml` | `LogPipeline` | Fluent Bit DaemonSet β†’ stdout JSON namespace `nova-invoice-suite` β†’ OTLP push | | `trace-pipeline.yaml` | `TracePipeline` | OTLP gRPC `:4317` from `@cap-js/telemetry` β†’ push | | `metric-pipeline.yaml` | `MetricPipeline` | Prometheus `/metrics:4004` + Istio + runtime β†’ OTLP | | `istio-telemetry.yaml` | Istio Telemetry | Envoy access logs forwarding | Tutti referenziano cert+key da secret `nova-cls-binding` via `valueFrom.secretKeyRef`. mTLS client validato. ### 10-ter.5 Ingestion setup Cloud Foundry CF firehose nativo: stdout dei pod CAP captato automaticamente da CF + routed a Cloud Logging via VCAP_SERVICES binding. **Nessun sidecar** richiesto. `@cap-js/telemetry` legge `ingest-otlp-endpoint` + `ingest-otlp-key` da binding credentials per traces/metrics. ### 10-ter.6 Access OpenSearch Dashboards Pre-IAS setup (default Cloud Logging built-in admin): 1. BTP Cockpit β†’ Cloud Logging instance β†’ **Dashboards URL** click. 2. Login con credentials da service key (`dashboards_username` + `dashboards_password`). 3. Index patterns disponibili (auto-create): - `logs-json-kyma*` β€” application logs - `logs-json-istio-envoy-kyma*` β€” Istio access logs - `traces-otlp-v1*` β€” distributed traces - `metrics-otlp-v1*` β€” application metrics ⚠️ **BTP-CLS-0001 critical recommendation**: dopo primo setup, abilitare SAML/IAS auth per dashboards (vedi [docs/runbooks/CLOUD_LOGGING_IAS_SETUP.md](runbooks/CLOUD_LOGGING_IAS_SETUP.md) β€” runbook dedicato). ### 10-ter.7 Cert rotation Cloud Logging Il secret `nova-cls-binding` contiene client cert + key per mTLS verso Cloud Logging endpoint. Rotation Γ¨ **on-demand** (non auto su BTP), procedura quarterly raccomandata. β†’ Runbook: [docs/runbooks/CLOUD_LOGGING_CERT_ROTATION.md](runbooks/CLOUD_LOGGING_CERT_ROTATION.md). ### 10-ter.8 Riferimenti - [`mta.yaml:355-386`](../mta.yaml) β€” CF binding + retention config + SAML template - [`k8s/telemetry/`](../k8s/telemetry/) β€” Telemetry CR + cls-instance.yaml - [`terraform/main.tf:123-128`](../terraform/main.tf) β€” entitlement Terraform - [`package.json:1187-1197`](../package.json) β€” `cds.requires.telemetry.kind = to-cloud-logging` - [`docs/explanation/06-logging-architecture.md`](explanation/06-logging-architecture.md) β€” design rationale + PII redaction + correlation IDs - [`docs/runbooks/CLOUD_LOGGING_IAS_SETUP.md`](runbooks/CLOUD_LOGGING_IAS_SETUP.md) β€” IAS/SAML dashboard auth setup - [`docs/runbooks/CLOUD_LOGGING_CERT_ROTATION.md`](runbooks/CLOUD_LOGGING_CERT_ROTATION.md) β€” cert rotation quarterly - [SAP Help Cloud Logging](https://help.sap.com/docs/btp/sap-business-technology-platform/what-is-sap-cloud-logging) - [BTP Security Recommendations BTP-CLS](https://help.sap.com/docs/btp/sap-btp-security-recommendations-c8a9bb59fe624f0981efa0eff2497d7d/sap-btp-security-recommendations?seclist-index=BTP-CLS&locale=en-US) --- ## 11. Troubleshooting deploy | Sintomo | Causa probabile | Risoluzione | |---|---|---| | `cf push` fail "no route" | Spazio quota memoria insufficiente | `cf scale nova-srv -m 1024M` o aggiungi quota | | Approuter 401 | XSUAA service not bound | `cf bind-service nova-approuter nova-uaa` + restage | | Approuter 502 Bad Gateway | nova-srv not healthy | `cf logs nova-srv --recent`; verify `/health` | | Kyma pod `ErrImagePull` | imagePullSecret mancante | Verifica registry pubblico o aggiungi `imagePullSecret` | | Kyma pod `CrashLoopBackOff` | Config error startup | `kubectl logs -p` (previous container); verify env vars | | `cds-serve` no .ts files | `CDS_TYPESCRIPT=true` mancante | Aggiungi a ConfigMap | | Action 501 NotFound | Handler non registrato | Verifica `srv/orchestratorSrv.ts` registra il handler | | OIDC JWT validation fail | Trust subaccount β†’ IAS non configurato | Re-establish trust Β§4.6 | | HANA HDI deploy fail | Schema migration in conflict | `cf delete nova-db-deployer`, re-deploy | | MTA build fail "module not found" | Dipendenza manca | `npm ci --legacy-peer-deps` prima di `mbt build` | Per altri issue: vedi [TROUBLESHOOTING.md](TROUBLESHOOTING.md). --- **Documentazione correlata**: - [ARCHITECTURE.md](ARCHITECTURE.md) β€” diagrammi topologia - [CONFIGURATION_GUIDE.md](CONFIGURATION_GUIDE.md) β€” env vars + parametri - [IAS_GROUPS_SETUP.md](IAS_GROUPS_SETUP.md) β€” dettaglio IAS - [cias-setup.md](cias-setup.md) β€” CIAS Scenario 1M1 - [cert-rotation.md](cert-rotation.md) β€” certificate rotation - [dr-runbook.md](dr-runbook.md) β€” disaster recovery - [runbooks/](runbooks/) β€” operational runbooks - [TROUBLESHOOTING.md](TROUBLESHOOTING.md) β€” problemi comuni