Structsure Console Technical Manual

Summary

Structsure Console is a web application implemented using the Node.js framework, Next.js. Users directly access the Structsure Console user experience through a web browser. Many user interactions result in REST-based API calls to Structsure Console’s REST API. The REST API, offered by Structsure Console, enumerates actions that users can take to effect change in both the console and the platform tools that are integrated with the console.

note

Structsure Console requires a dedicated Postgres database and Keycloak instance.

Configuration

Structsure Console takes configuration largely in the form of environment variables consumed by the Node.js process. Here are those environment variables|

Environment Variable	Description
ARGOCD_BASE_URL	The URL for Argo CD. Structsure Console currently supports only one Argo CD instance
CONFLUENCE_BASE_URL	The URL for Confluence.
CONFLUENCE_PERSONAL_ACCESS_TOKEN	PAC structsure-console uses for auth for all Confluence API interactions
DB_HOST	hostname of database server, like localhost (not required if DATABASE_URL is set)
DB_PORT	network port for database server, like 5432 (not required if DATABASE_URL is set)
DB_USER	username that the structsure-console server will authenticate as when communicating with the database (not required if DATABASE_URL is set)
DB_PASS	password that the structsure-console server will authenticate as when communicating with the database (not required if DATABASE_URL is set)
DB_NAME	the name of the logical database containing structsure-console's data (not required if DATABASE_URL is set)
ENABLE_CONFLUENCE	“true” or “false” to denote whether Confluence support is enabled
ENABLE_GITLAB	“true” or “false” to denote whether GitLab support is enabled
ENABLE_JIRA	“true” or “false” to denote whether Jira support is enabled
ENABLE_MATTERMOST	“true” or “false” to denote whether Mattermost support is enabled
DATABASE_URL	database connection url, like postgres://db_user:db_pass@db_host:1234/db_name?schema=public (this is the default method used for database connection url, you can use this or the deconstructed variables above)
GITLAB_BASE_URL	The URL for GitLab. Should not include anything to the right of the complete hostname and optionally the port GITLAB_LOGO The logo presented in the UX for GITLAB_PERSONAL_ACCESS_TOKEN
JIRA_BASE_URL	The URL for Jira.
JIRA_PERSONAL_ACCESS_TOKEN	The PAC Structsure Console will use to authenticate all comms with JIRA
JIRA_USERNAME	The Username of the user owning the JIRA_PERSONAL_ACCESS_TOKEN.
KEYCLOAK_BASE_URL	Note that it should always end in /auth
KEYCLOAK_ID	The Client ID for the Structsure Console client configured in Keycloak
KEYCLOAK_MANAGEMENT_CLIENT_ID	A UUID; find it by visiting Keycloak admin console, clicking Clients, then realm-management. The UUID is the last portion of the URL in your address bar.
KEYCLOAK_REALM	The realm name where Structsure users will exist.
MATTERMOST_BASE_URL	The URL for Mattermost.
MATTERMOST_PERSONAL_ACCESS_TOKEN	Personal access token with admin privileges
NEXTAUTH_SECRET	A random string is used to hash tokens, sign/encrypt cookies and generate cryptographic keys

Initialization

After a fresh installation of Structsure Console, the application needs to perform some initialization of its database. This initialization needs to be triggered by an actor in the Keycloak realm being used for OIDC with Structsure Console. Moreover, that actor needs to have realm-management roles/permissions in Keycloak.

Initialization Steps

1. Visit the {console-url}/api/initialize endpoint in a browser to initialize the database with the necessary tools.
2. Visit {console-url} in a browser and log in as the user with realm-management permissions
3. Visit {console-url}/api/initialize again. This time, you will be established properly in Structsure Console as a Structsure Admin.
4. Restart the structsure console processes.

Authentication

Structsure Console defers authentication to an OpenID Connect provider (OIDC provider). More specifically, Structsure Console relies on an integration with a Keycloak instance in order to authenticate users. NextAuth.js is used for the OIDC interactions with Keycloak. When processing a subsequent request, Structsure Console will use the resulting JWT from a successful authentication to determine who the current actor is.

Authorization

Structsure Console has a fine-grained enumeration of possible user actions. Each of those user actions has corresponding application logic to determine if an actor is allowed to perform the action. The list of actions, along with their associated logic, comprises the internal Authorization Service for Structsure Console.

For some user actions, Structsure Console benefits from Keycloak as a redundant validation of action authorization.

Platform Tool Integration

As mentioned, Structsure Console interacts with other tools in the platform using HTTP-based APIs offered by those tools.

Example Action Execution

Many actions performed by a Structsure Console user trigger these integrations between Structsure Console and one-or-more platform tools. The following example shows how Structsure Console handles the Create User action, which can be performed by a Structsure Admin. Note that each numbered step happens sequentially.

1. Confirm the actor is authorized to perform the Create User action using the internal Authorization Service.
   1. If not authorized, fail.
2. Create the user in Keycloak using the Keycloak Admin API.
   1. If unsuccessful, fail.
3. Create the user in the application database.
4. In parallel, create the user in each Platform Tool.
   1. If the action fails in any of the platform tools, do not fail the overarching action.
5. Respond to the client (web browser) appropriately based on success of the action.

There are no attempts to retry any failed actions at any level in the above flow. In the case of partial success, Structsure Console offers some features like “Repair Organization” and “Repair User” to address partial failures (please see the User Manual).

Keycloak

As mentioned earlier, Structsure Console relies on Keycloak to provide authentication via the OIDC protocol. However, Structsure Console also uses Keycloak’s admin REST API to represent Structsure concepts.

Concept Mapping

Structsure Concept	Keycloak Concept
Organizations	Top-level groups
Roles	Subgroups with specific names
Users	Users

GitLab

Structsure Console integrates with GitLab to define permissions boundaries for Organizations, Teams, and Users. GitLab’s REST API is used for this integration.

Concept Mapping

Structsure Concept	GitLab Concept
Organizations	Top-level groups
Teams	Subgroups
Roles	Roles
Users	Users

Jira

In addition to user management, Structsure Console uses Jira’s REST API to create Projects with proper scoping and permissions. Such Jira Project creation can be invoked in the Project Tools user experience. Note that the Jira integration supports the Structsure Admin role (assigning a user that role would give them Jira admin privileges).

Concept Mapping

Structsure Concept	Jira Concept
Organizations	Couplings of a User group with a Permission Scheme
Users	Users

Confluence

Structsure Console implements a very minimal integration with Confluence. Confluence is only supported when Jira is deployed and Confluence must be configured to sync users from Jira. The Confluence integration supports the Structsure Admin role; assigning a user the Structsure Admin role gives them Confluence admin privileges.

Concept Mapping

Structsure Concept	Confluence Concept
Organizations	Spaces (no automation of permissions today)

Mattermost

Concept Mapping

Structsure Concept	Mattermost Concept
Organizations	Teams
Users	Users

Argo CD

Argo CD relies entirely on its configured identity provider to determine users and permissions. Therefore, Console does not need to implement any automation related to user management. Structsure Console will query Argo CD for projects and applications to present to users within Console's Projects user experience. Console relies on some naming conventions for the integration to work properly.

Concept Mapping

Structsure Concept	Argo CD Concept
Applications	Applications
Organizations	User groups claimed in JWT
Projects	Projects
Users	Users

Identity

Structsure Console interacts with Argo CD on behalf of the authenticated actor. More specifically, Structsure Console presents the current session's access token to the Argo CD REST api whenever such an interprocess communication is necessary. For this integration to work, a few things need to be configured properly in Structsure's Keycloak.

1. The access token's content must include the actor's group memberships.
2. Specify that Argo CD is an intended consumer of Structsure Console's access tokens.

Configuring Structsure Console's Access Tokens:

1. Log in to the Keycloak Admin Console, in the structsure realm.
2. Click Clients in the left menu.
3. Find Structsure Console's client, and click on it.
4. Click on the Client scopes tab.
5. Click on the ...-dedicated client scope in the list.
6. Click the Add mapper button, then From predefined mappers.
7. Find groups, check its box, and click Add.
8. Click the Add mapper button, then By configuration.
9. Click the Audience entry in the list.
10. In the Included Client Audience field, select argocd.
11. Toggle Add to ID token to On.
12. Toggle Add to access token to On.
13. Click Save.

Conventions

Structsure Console currently relies on Structsure Admins to create projects and applications in Argo CD by whatever means they prefer. However, some naming conventions are necessary in order to associate Argo CD projects with Structsure Projects. The naming convention is as follows:

<organization name>-<project-name>

For any Argo CD project created that corresponds to a Structsure project, the Argo CD project should be named by the convention above. The project name should be lowercase and kebab case, so that if we have an organization named Structsure, and a project named Structsure Console, the associated Argo CD project name should be:

structsure-structsure-console

This convention supports discovering and associating applications at runtime. Note that the current Structsure Console user must have permission to access the relevant project within Argo CD in order to see the applications in Console.