Set up a team in Sysdig Monitor

Sysdig Monitor provides system-level monitoring of Kubernetes hosts and the ability to create custom dashboards, alerts and operational-level captures to diagnose application or platform-level issues.

The Sysdig Teams Operator runs in the cluster and enables a team to create and manage access to a dedicated Sysdig Team account for BC Gov Private Cloud PaaS users. The team is scoped to the OpenShift namespaces that belong to the team. Sysdig also provides a default dashboard to identify system resources, limits and actual usage.

For more information on Sysdig Monitor, see Monitoring with Sysdig.

Go to the BCDevOps Sysdig Monitor.
- Alternatively, you can also sign in on the Sysdig site. Select OpenID and type BCDevOps as the company.
Upon login, you will be presented with a default page. You may be directed to the Catchall Team which has access to no resources at the moment (the team resources and access will be created in later steps).
At the bottom left corner of the default page, you can find the initial icon for your account and the email address associated with it.
- Note: Sysdig identifies users by the email, so it's important to use the correct email address for yourself as well as your team members.

Create Sysdig team access

The OpenShift Operator runs in the background and creates a Sysdig team RBAC and dashboard for you. The operator looks for a sysdig-team custom resource from your *-tools namespace. There are two parts of work to create Sysdig team access.

Part 1 - Compose the sysdig-team object manifest

Below is a sample of the sysdig-team custom resource manifest. We also provide a custom resource manifest template file you can use.

Name and Namespace: The sysdig-team custom resource should be created in your OpenShift project set tools namespace. Name it using the license plate number for your project set to make sure it's unique.

Team Description: Add a description for the sysdig team.

Team Users: A list of users to be added to this team.

User Name - Sysdig identifies users by the email address, so make sure everyone on your team logs in to Sysdig and obtains the correct email from their Sysdig User Profile.
User Role - These are the available roles to assign to different team members:
- ROLE_TEAM_EDIT (Advanced User): Read and write access to the components of the application available to the team. Can create, edit and delete dashboards, alerts or other content. Recommended for administrators and team members that need to create and manage dashboards.
- ROLE_TEAM_STANDARD (Standard User): Same as Advanced User but without access to the Explore page. Recommended for developers that need to refer to dashboards for resource tuning and service monitoring.
- ROLE_TEAM_READ (View-only User): This role has read-only access to the environment in the team scope. Recommended for project owners and team members from the business area.

Part 2 - Create the sysdig-team custom resource

Note:

Only apply role updates to the custom resource from tools namespace. Do not use the Sysdig Monitor UI to modify team access, because the operator reconciliation will overwrite any UI changes to team roles and settings.

If your project set is on Gold and GoldDR clusters, only create the sysdig-team custom resource in the Gold cluster. The Sysdig operator can create the dashboards for your applications across both clusters.

It's important to keep a single and unique sysdig-team custom resource per project set because you don't want to have duplicated teams on Sysdig. Please make sure to remove sysdig-team custom resource from dev, test and prod namespaces if they are created accidentally.

So now you have the sysdig-team manifest file ready, use oc apply and let the operator create the team on Sysdig.

# switch to the tools namespace
oc project <PROJECT_SET_LICENSE_PLATE>-tools
# edit the sample sysdig-team resource and apply the manifest
oc apply -f sysdig-team-sample.yaml

Verify Sysdig team creation

Use oc describe sysdig-team <PROJECT_SET_LICENSE_PLATE>-sysdigteam to validate that the Sysdig team was created:

Name:         101ed4-sysdigteam
Namespace:    101ed4-tools
Labels:       <none>
API Version:  ops.gov.bc.ca/v1alpha1
Kind:         SysdigTeam
Metadata:
  Creation Timestamp:  2021-04-15T22:42:20Z
  ...
Spec:
  Team:
    Description:  The Sysdig Team for the Platform Services Documize
    Users:
      Name:  shelly.han@gov.bc.ca
      Role:  ROLE_TEAM_MANAGER
      Name:  patricksimonian@gmail.com
      Role:  ROLE_TEAM_EDIT
      ...
Status:
  Conditions:
    Ansible Result:
      Changed:             0
      Completion:          2021-08-18T20:10:43.665524
      Failures:            0
      Ok:                  30
      Skipped:             13
    Last Transition Time:  2021-08-05T18:54:24Z
    Message:               Awaiting next reconciliation
    Reason:                Successful
    Status:                True
    Type:                  Running
Events:                    <none>

You should expect to see the following from the sysdig-team outcome:

Message:               Awaiting next reconciliation
Reason:                Successful

If both of these show, the sysdig-team custom resource is processed successfully. You can go back to Sysdig to see the new team scope and default dashboards.

To access them:

Log in to Sysdig like how you did just now.
Navigate to the bottom left hand of the page to switch your team, which should be named as [PROJECT_SET_LICENSE_PLATE]-team.
You may need to wait some time between the creation of the team and resources to display.

Review your monitoring dashboards

You should see the following dashboard templates from your Sysdig team:

A resource dashboard template provides an overview of resource allocation for production namespace. You can make copies of it for different environments.
A series of Sysdig predefined dashboards exist for general use or to help teams create custom dashboards. Click Dashboards, then click Add Dashboard and Create from Template to pick a template to use.

Note: The Platform Services team recommends that you use the Sysdig API to keep your dashboards as code. Each dashboard is assigned to an account on Sysdig for ownership. If you delete the user (whether from the console or custom resource), all of the dashboards are deleted. Sysdig Cloud (SaaS) does not provide a service to retain the deleted dashboards for a user.

Troubleshooting

Error from sysdig-team custom resource: if you don't see Awaiting next reconciliation after waiting for 5 minutes, contact the Platform Services team on the #devops-sysdig Rocket.Chat channel. Make sure to include the OpenShift cluster and namespace information.
If you don't see the Sysdig team created, please double check that: -sysdig-team custom resource is created in tools namespace
- There are no duplicated sysdig-team custom resources in dev/test/prod namespaces. Please run oc -n <NAMESPACE> delete sysdig-team <SYSDIG-TEAM-NAME> to delete the extra custom resource.
- Your Sysdig account profile matches the email address that you have provided in the sysdig-team custom resource. If there is a mismatch, reapply the custom resource.
If you don't see a default dashboard in your Sysdig team, contact the Platform Services team on the #devops-sysdig Rocket.Chat channel.