Skip to main content
Version: 5.17.0

Release Notes

5.17.0 (2024-08-16)​

This release of Structsure Enterprise v5.17.0 includes BigBang Version 2.32.0. For more details on the features and updates included in BigBang Version 2.32.0, please refer to the BigBang release notes.

Upgrade Notices​

Kubernetes Upgrade Requirement​

This release requires Kubernetes native sidecars. Hence, an upgrade to Kubernetes v1.29 or later is required.

Post-Upgrade Instructions​

To ensure the istio-proxy sidecar container switches to an init container (due to Kubernetes native sidecar support), restart all Istio-supporting pods immediately after the upgrade. Use the following commands:

for ns in $(kubectl get ns -l app.kubernetes.io/part-of=bigbang,istio-injection=enabled -o custom-columns=":metadata.name"); do
  kubectl rollout restart deployment -n $ns
  kubectl rollout restart statefulset -n $ns
  kubectl rollout restart daemonset -n $ns
done

Backup Recommendations​

It is highly advised to create backups of current admin passwords before performing the Structsure upgrade for the following applications to prevent the loss of credentials:

  • Grafana
  • Keycloak
  • SonarQube
  • NeuVector

Default Admin Passwords​

If not changed manually, the default admin passwords are as follows:

  • Grafana: Monitoring-grafana secret within the monitoring namespace. This password will be overridden to a randomly generated strong password.
  • Keycloak: keycloak-env secret within the keycloak namespace. In tested upgrades, the admin password didn’t change when only the composition was applied, but it may change during the full upgrade with the Structsure Zarf package due to changes in the default chart values.
  • SonarQube: Default username and password are both admin. The password is prompted to change upon first login. Similar to Keycloak, the password may change during the full upgrade.
  • NeuVector: Default username and password are both admin. The admin and metrics users' passwords will be overridden to randomly generated strong passwords.

Major Features​

Persistent Volumes for Monitoring Applications​

We now offer persistence via PVCs (Persistent Volume Claims) for the following monitoring applications:

  • Grafana
  • Prometheus
  • Alertmanager

Enabling Persistence​

To enable persistence, set persistence = true in the cluster_inputs object in your hcl and rerun your Terragrunt stage. Caution: If you are currently overriding, providing custom PVCs, or using a third-party solution for these applications, your settings might be overwritten.

Specific Considerations​

  • Grafana: When persistence is enabled, only one pod is allowed due to the ReadWriteOnce access mode of the default EBS storage class. Refer to the docs to configure ReadWriteMany (e.g., Amazon EFS) if multiple pods are desired.
  • Prometheus: Allows for adjusting the retention period and retention file size.

Compatibility​

Zarf Version​

The packages for this release were built using the following Zarf version:

  • Zarf: v0.32.6

Kubernetes Distributions and Versions​

The packages were tested across the following Kubernetes distributions and versions:

  • Rancher Kubernetes Engine 2 (RKE2): v1.29.7+rke2r1
  • Kubernetes Lightweight (K3S): v1.30.0
  • Elastic Kubernetes Service (EKS): v1.29

AMI Versions​

The following AMI versions were used for testing:

  • RKE2 AMI: structsure-rke2-v1.29.7-rke2r1-rocky-8-base-v1.1.1-stig-2024-08-12T08-14-46Z
  • EKS AMI: structsure-eks-1.29.6-rocky-8-base-v1.1.1-stig-2024-07-29T08-12-23Z
  • Base AMI: Rocky-8-EC2-LVM-8.10-20240528.0.x86_64

Changelog​

IaC Bug Fixes​

  • iac: always set s3 regionendpoint for zarf init
  • iac: automatically remove duplicates in allowed_security_groups

Package Features​

  • bump console version to 5.52.x
  • crossplane: argocd declarative sso
  • crossplane: Neuvector is configured with read-only prometheus user and admin user with random passwords
  • crossplane: web app default passwords randomly generated
  • increment rke2 k8s to 1.29, fail zarf package deploy if k8s version < 1.29
  • persistent storage for monitoring apps
  • upgrade Big Bang to v2.32.0

Package Bug Fixes​

  • dig keycloak config realm to avoid nil pointer during zarf deploy
  • prevent kyverno policy from erroneously helm templating values
  • revert rendering of values in structsure-enterprise chart
  • use appropriate whitespace in generate kyverno policy

Documentation​

  • updated zarf version in documentation
  • Check out the documentation for guidance
  • BigBang Release Notes

Warning​

It is Highly advised to create backups of current admin passwords before this smoothglue upgrade of the following apps to ensure protection of lost credentials. Implementing this upgrade may change the admin passwords to randomly generated strings for the following listed apps. The new passwords can be found and overridden within this docs page: https://structsure.gitlab.io/-/jigsaw/structsure-enterprise/-/jobs/7548192629/artifacts/public/docs/unreleased/how-to/operations/how-to-change-app-credentials/

Once the full release is cut they can be found here: https://docs.structsure.io/docs/unreleased/how-to/operations/how-to-change-app-credentials/

The Default admin passwords are the following if they haven’t been changed manually:

  • Grafana (Monitoring-grafana secret within monitoring namespace. This admin password WILL be overridden to a randomly generated strong password.)

  • Keycloak (keycloak-env secret within keycloak namespace.)

  • Sonarqube (Default username is admin and so is the password, prompted to change it upon first login.)

  • Neuvector (Default username is admin and so is the password. This admin and metrics users' passwords WILL be overridden to a randomly generated strong password.)

5.17.0-rc.1 (2024-08-09)​

Package Features​

  • bump console version to 5.52.x
  • crossplane: argocd declarative sso
  • crossplane: Neuvector is configured with read-only prometheus user and admin user with random passwords
  • crossplane: web app default passwords randomly generated
  • persistent storage for monitoring apps
  • upgrade Big Bang to v2.32.0

Package Bug Fixes​

  • dig keycloak config realm to avoid nil pointer during zarf deploy
  • prevent kyverno policy from erroneously helm templating values

Documentation​

  • add self to maintainers, update onboarding template with devenv setup link
  • update zarf version

5.16.0 (2024-07-25)​

Warning​

  • Included in this update is a fix for crossplane pods to be excluded from a kyverno policy that was blocking scheduling for crossplane pods. The fix will be applied automatically as part of the package. The IaC creates some values for kyverno policies that are no longer needed as those have been moved to the package. If you upgrading from a previous version, you will need to review the kyverno-policies-overrides config map in the structsure-system namespace and remove any entries that the IaC added. IaC specific entries can be identified by viewing the value file outputs of the IaC.
  • Included in this update is better support for a containerd mirror of registry1.dso.mil to the internal zarf registry. Running the IaC to apply these changes will generate a new launch template and will cycle EKS nodes onto the new config. If you are using a Structsure EKS AMI, you will need to use an AMI that was built on or before 07/11/2024. If using an AMI prior to that build date, the containerd mirror will stop functioning until a new AMI is used.

IaC Bug Fixes​

  • iac: eks registry mirror for all environments

Package Features​

  • enable NetworkPolicy in vpc cni by default
  • Upgrade Big Bang to v2.31

Documentation​

  • don't render wiki/style guide as part of public docs site

5.15.0 (2024-07-11)​

Warning​

Changes to Console password generation require Terraform to generate a new password for pre-existing deployments. As a result, Console will be down from the time the IaC is ran until the cluster has reconciled the outputted BigBang values files.

Additional Notes​

As part of the Big Bang upgrade, Keycloak may have some trouble reconciling. We have tested that a way to ensure a smooth upgrade is to, before upgrading, delete the statefulsets for Keycloak in your cluster and let the new version upgrade the helm to bring it back. For more notes on this, please see the official documentation for Big Bang 2.30.

Also, Kiali is now enabled by default.

IaC Features​

  • iac: EKS containerd mirror for ironbank images
    • For now, this is supported for Structsure AMIs. We have noticed that non-Structsure AMIs have an issue with the config and we are working on a follow-up solve.
  • iac: eks node group sizes as obj var
    • Users can now set the sizes (min, max, desired) of the default EKS nodegroup via hcl file.
  • iac: rds engine_version as var
    • Users can now set the versions of each modules RDS via {tool}-inputs in the hcl file. Note that users who have their Confluence RDS database version greater than 13.8 will revive an error upgrading infra until this release when they can set the RDS database version.
  • iac: Save SSH private keys to AWS Parameter store from SSM
    • The SSH key will now be found in the AWS Secrets Manager instead of the SSM Parameter store.
  • iac: update default iac k8s version to 1.28

IaC Bug Fixes​

  • iac: special characters within console database password

Package Features​

  • crossplane: automatic kiali sso integration
  • Enable docs search
  • Upgrade Big Bang to 2.30.0

Package Bug Fixes​

  • crossplane: crossplane exempt from kyverno drop all policy

Documentation​

  • new section on High Availability Configuration

5.14.1 (2024-07-01)​

Additional Notes​

Starting with Gitlab 17, runner registration tokens are no longer supported by default. Applying this version as an upgrade to a pre-existing cluster will require either re-enabling runner registration tokens (https://docs.gitlab.com/ee/administration/settings/continuous_integration.html#enable-runner-registrations-tokens) or creating a runner authentication token and updating the gitlab-gitlab-runner-secret secret in the gitlab-runner namespace with the new token (https://repo1.dso.mil/big-bang/product/packages/gitlab/-/blob/main/docs/gitlab17.md)

If SSO configuration has already been setup manually, disregard this. However, if you are using the new automated SSO configuration features, any crossplane-managed keycloak groups will need to be reassociated to their crossplane object. To do so, find groups that are currently failing by kubectl get groups.group.keycloak.crossplane.io and for any that are Synched = False, you will need to manually get the UUID from keycloak and apply the crossplane.io/external-name: <UUID> annotation to the object.

Package Bug Fixes​

  • zarf: upgrade gitlab to 17.1.1 to patch cve

5.14.0 (2024-06-27)​

Additional Notes​

If SSO configuration has already been setup manually, disregard this. However, if you are using the new automated SSO configuration features, any crossplane-managed keycloak groups will need to be reassociated to their crossplane object. To do so, find groups that are currently failing by kubectl get groups.group.keycloak.crossplane.io and for any that are Synched = False, you will need to manually get the UUID from keycloak and apply the crossplane.io/external-name: <UUID> annotation to the object.

IaC Bug Fixes​

  • iac: set cluster_iam_role_dns_suffix in EKS module
  • iac: set preserve_client_ip to null if compatibility_mode is true

Package Features​

  • monitoring apps sso with xrds
  • bump console to 5.49.20240614000040
  • console: update console to 5.50.x
  • crossplane: grafana uses keycloak XRD provider
  • crossplane: Keycloak-config is fully configurable from values.yaml
  • crossplane: Neuvector declarative configuration
  • Keycloak CVE fix
  • update console to 5.49x and chart 1.7.0
  • update to include console 5.49.20240614180839
  • Upgrade Big Bang to 2.29.0

Package Bug Fixes​

  • crossplane: Keycloak-Config reports ready
  • crossplane: secret management for keycloak-config

Documentation​

  • Create Release Notes Section in our Documentation

5.13.0 (2024-06-11)​

Package Features​

  • console: upgrade console to use v1.5.1 chart
  • update console to latest v5.48 to support ami
  • upgrade big bang to 2.28.1

Package Bug Fixes​

  • nest templating of keycloak hostname
  • url syntax within structsure chart

5.12.0 (2024-06-04)​

IaC Features​

  • iac: nexus iac

IaC Bug Fixes​

  • iac: IAC logic to handle no zarf registry s3 backing

Package Features​

  • console: include latest Console v5.47
  • crossplane: Create XRD and Zarf package for Nexus
  • update console to support keycloak 23
  • upgrade big bang to 2.27.0

Package Bug Fixes​

  • docs: transitory 404s on docs site
  • metrics-server not being deployed automatically in EKS clusters
  • zarf,crossplane: CVE fix for Confluence and Jira
  • zarf: ingress-pki failures related to Vault if CA_CERT is not specified

Documentation​

  • update docs to include keycloak-config zarf variable
  • zarf: fix example zarf config in docs

5.11.1 (2024-06-17)​

🚨 Danger 🚨​

This release has a significant refactor in how configmaps/secrets for the Bigbang HR are created. Crossplane will recreate all of the ones Structsure manages which can result in applications temporarily being undeployed. To prevent problems and outages, suspend the bigbang helm release when performing the upgrade for this Structsure release. Afterwards, you can verify configmaps/secrets for the bigbang helm release are present and can unsuspend the helm release.

Package Bug Fixes​

  • add kyverno policy exception for promtail capability

5.11.0 (2024-05-15)​

🚨 Danger 🚨​

This release has a significant refactor in how configmaps/secrets for the Bigbang HR are created. Crossplane will recreate all of the ones Structsure manages which can result in applications temporarily being undeployed. To prevent problems and outages, suspend the bigbang helm release when performing the upgrade for this Structsure release. Afterwards, you can verify configmaps/secrets for the bigbang helm release are present and can unsuspend the helm release.

IaC Features​

  • iac: add efs-dynamic iac module for RWX storage

Package Features​

  • configuring ArgoCD for high availability
  • create dashboard for daily active users in grafana
  • crossplane: add keycloak configurations managed by XRD
  • upgrade big bang to 2.26.0
  • zarf deployment waits for all packages to report ready status

Package Bug Fixes​

  • promtail: allow promtail to bypass read access control
  • zarf: no minio-overrides cm or secret created when deploying minio

Documentation​

  • adding edge install and maintenance docs to docusaurus
  • how to use custom images / git repos
  • inline code block styling
  • Structsure doc edits

Other Changes​

  • iac: remove unused variable declarations from iac
  • xrd: convert application XRDs for function pipelines

5.10.0 (2024-05-01)​

IaC Bug Fixes​

  • iac: gitlab tmp bucket

Package Features​

  • Upgrade BB to 2.25
  • Vault IaC and HA config

Package Bug Fixes​

  • patch nfs permission fixer for Confluence

Documentation​

  • Argo CD is two words

5.9.0 (2024-04-16)​

IaC Features​

  • iac: adding create IAM role logic to Gitlab, Mattermost, Loki, and Velero
  • iac: Enable RKE2 customer supplied userdata config

IaC Bug Fixes​

  • iac: Add policy for S3 Userdata
  • iac: update cp and va var names

Package Features​

  • crossplane: patch, go-templating functions & keycloak-provider
  • enable keycloak fine grained authz by default
  • Upgrade BigBang to 2.24

Documentation​

  • web proxy configuration

5.8.0 (2024-04-05)​

Additional Notes:​

  • Minimum zarf version supported is now v0.32.6

IaC Features​

  • iac: Add efs for jira and confluence
  • iac: add IRSA support for Velero add-on
  • iac: Update EKS IAC to allow clients to provide an existing IAM role

IaC Bug Fixes​

  • iac: EKS IAM policies not tagged when compatibility_mode is set

Package Features​

  • upgrade console to 5.45x
  • upgrading to Big Bang 2.23.0

Package Bug Fixes​

  • config from previous deploys can't be unset
  • zarf: cluster auditor claim/secret name
  • zarf: on init disable storage redirect
  • zarf: resolve issue with helm templates in multi-node clusters

Documentation​

  • add compatability mode documentation

Other Changes​

  • iac: add IRSA related variables to EKS IaC

5.7.0 (2024-03-20)​

IaC Bug Fixes​

  • iac: offering other IAM role than InstanceOpsRole

Package Features​

  • update console to 5.44
  • upgrading Big Bang to 2.22.0
  • zarf: upgrade supported zarf version to v0.32.4

Package Bug Fixes​

  • upgrade Confluence to 8.8.1 to address vulnerabilities

Documentation​

  • add Identity section to Console technical manual
  • clarify release to main branch instructions
  • commit/MR guidelines for release notes

5.6.0 (2024-3-5)​

Warning:​

  • Changes to Sonarqube password generation require terraform to generate a new password for pre-existing deployments. As a result, Sonarqube will be down from the time the IaC is ran until the cluster has reconciled the outputted BigBang values files.
  • Changes to GitLab's RDS settings will cause a restart of the database. GitLab will be down during the restart.

Additional Notes:​

  • With the addition of cluster-autoscaler, the recommended minimum version of k8s is now 1.27.

Features​

  • add ability to force SSL on Gitlab RDS, on by default
  • add cluster-autoscaler xrd and zarf packages
  • add Kiali as optional add-on
  • add Tempo as optional add-on
  • enable keycloak cac auth support
  • update Crossplane provider-kubernetes to v0.11.4
  • update Crossplane to v1.15.0
  • Upgrade BigBang to v2.21
  • upgrading Big Bang to 2.21.1

Bug Fixes​

  • sonarqube password generation

Other Changes​

  • document the release process
  • docusaurus design updates
  • minor corrections
  • minor updates
  • simplify IaC CI
  • versioning the 5.6.0 documentation

5.5.0 (2024-2-21)​

Features​

  • add metrics-server BigBang add-on
  • allow shared terragrunt modules outside of git repo
  • bb 2.20.0 upgrade
  • enable automatic etcd backup to s3
  • update structsure console to 5.43
  • use differential zarf packages to speed up package build and deploy

Bug Fixes​

  • add ability to turn off NLB security groups
  • exporting user data variables
  • force re-creation of crossplane private-registry-internal secret

Other Changes​

  • allow creation of pre-release on main even if mr is open
  • alternative method to SSH into a cluster
  • blackhole api.github.com in coredns EKS add-on
  • landing page updates
  • versioning the 5.5.0 documentation

5.4.1 (2024-2-12)​

Bug Fixes​

  • add missing redis image used by authservice
  • don't hardcode rke2 userdata region
  • ensure RKE2 AMI filter selects the Rocky 8 AMI
  • handle empty ami data object in RKE2 IaC
  • remove link
  • update checksum var

Other Changes​

  • add note about missing Authservice image to upgrabe-big-bang doc
  • always build iac bundle on tags and protected branches
  • disable release jobs for branch and MR pipelines
  • refactor documenation pipeline to fix pages and publishing

5.4.0 (2024-2-9)​

Features​

  • add docs and IAC package to release pipeline
  • update provider-kubernetes to v0.11.2
  • update Structsure Console to 5.42.20240202021119
  • upgrading big bang to 2.19.1

Bug Fixes​

  • checksum-manifest only included on tagged commits
  • pipelines cannot retrieve latest zarf release package
  • restrict the EKS terraform module to version 19.x
  • rke2 CP / kyverno

Other Changes​

  • add instructions for deploying trend micro
  • create missing versions
  • fix error in release-zarf-package job in main
  • govcloud release & checksum release
  • merge Structsure AWS IaC repo into Structsure Enterprise
  • push final docs to docs.structsure.io
  • quickstart: remove duplicate quickstart
  • update single node install instructions

5.3.0 (2024-1-24)​

Features​

  • add govcloud runner & add runner tags
  • checksum manifest for zarf assets
  • update to Big Bang 2.18.0

Other Changes​

  • automatically link and upload packages to s3 on tag pipelines
  • console: initial Structsure Console docs
  • remove all requests to api.github.com

5.2.0 (2024-1-19)​

Features​

  • update console image to 5.42.20240111143003
  • update to Big Bang 2.17.0

Bug Fixes​

  • update crossplane version checks in zarf package
  • update neuvector tag to fix bug
  • upgrade Confluence to 8.7.2 to address vulnerabilities

Other Changes​

  • add separate build-zarf-sbom job
  • automatically upload release artifacts to s3
  • don't attach artifacts with semantic-release
  • don't publish sbom if one was not generated
  • increase pipeline instance storage size
  • refactor build jobs as part of parent pipeline
  • revert separate job for zarf sbom