Notes on “DevSecOps Worst Practice by Tanya Janca”

Recently I watched a talk from Tanya Janca on DevSecOps worst practices. The talk is interesting because I have seen similar practices during the early adoption stage of DevSecOps processes and culture. If you find that your DevSecOps practices have plateaued, chances are that your program is practicing some of the bad practices.

Below is a list of bad practices mentioned by Tanya:

1. Breaking Build on False positives
2. Turning on a tool without testing
3. Artificial Gates
4. Missing Test Results (Results are not shared in a convenient location)
5. Missing Runway Testing (Not testing the tools in the CI pipelines to see the job duration impact)
6. Impossible SLAs
7. Untrained Staffs
8. Bug lost in Backlog
9. No Positive Reinforcement
10. Only worrying about your part (Not understanding whether another teams are facing any issues if you make the changes)
11. Multiple Bug trackers (Bugs tracking are scattered in different dashboards)
12. INSecure SDLC (Tools are not enough. We need more process and other activities like Threat Modelling)
13. Overly Permissive CI/CD (Skipping a process to deploy to prod)
14. Automation only in CI/CD
15. Hiding Mistakes and errors

I will only highlight a few of these practices and share my thoughts.

Turning on a security tool without thorough testing

Testing a new tool for effectiveness is HARDER than we think. Usually POC on a vendor product has a limited timeframe (about one month plus). And you need to identify the teams that are suitable to test the products. If you chose the wrong team to test the product, you might jump into conclusion that this product is bad. Or if you test it on only subset of team, we still find it uncertain if the performance can be replicated to other teams who are using a different tech stack.

Tanya mentioned that one way we can do is buy perpetual license and trial it for about 6 months before deciding to purchase a larger contract. I think this is a good recommendation.

From vendor point of view, you should think about how you can let the security team try the important features of product and test it sufficiently so that they can make better conclusions.

Overly Permissive CI/CD

Tanya gave an example about how the development team can disable tests in order to deploy to production. If too much permission is given to the team, some might abuse the permissions to reach their goal.

In general, I think CI is a wild jungle where you can run any scripts or workflows. But the crucial thing is to restrict the permissions for deployment (CD). If we control the permissions better, we can stop people from abusing deployment rights.

Impossible SLAs

In some organizations, there are strict SLA requirements for deployment to production. For example, there can be a SLA that “NO CRITICAL or HIGH” issues in production.

This might be impossible to meet for some of the teams (Especially if they have legacy apps). Tanya recommends that Security team should consult other teams to see if this SLAs is feasible. Listen to see if the teams can fix the issues to meet the SLAs.