
Georges (Community Member) asked a question.
β
I set up a dashboard to track the time-to-fix per application per severity, filtering out irrelevant data using a custom filter. Please see screenshot attached with application names obfuscated for obvious reasons (for your clarity: same color for rows belonging to one app).
β
Issue: For any given application, the data in this dashboard doesn't match the numbers of flaws when I go to "Triage Flaws" module for that application and apply the same filters. The orange example from the screenshot shows 1 Very High, 17 Highs and 163 Mediums in the dashboard, while the Triage Flaws module shows 0 Very Highs, 1 High and 111 Mediums (see other screenshot attached)
β
Can anyone shed a light as to what am I missing here?
Best regards,
Georges
β
.png)
Second screenshot mentioned in the original post
P.S.: I'm inclined to believe the numbers from "Triage Flaws" are correct as they match those on the "Results : Latest" page
Hi Georges,
It seems to be because the searches in Analytics and Triage Flaws are for different logical buckets of flaws:
If it's helpful, I'll also share how my team tracks TTR:
From here, you can apply additional filters for only certain applications, only certain severities, etc.
We find displaying the data this way tells a better narrative to our stakeholders than all-time averages - it shows how our teams are doing over time and whether their TTRs for flaws closed that month are getting shorter or longer.
Hope all this helps!
Cheers,
Kate
Hello Kate and thank you so much for this detailed answer! This indeed helps a lot π
β
> Are you trying to find Time to Resolve (TTR) of open flaws?
β
Indeed, I'm looking for the Veracode estimates and use that number to roughly see which teams are drowning in their backlog the most so we can allocate resources appropriately. I was not aware that there was a boolean field tracking the open-ness of a finding, so I added that now to replace the "Mitigation Status" filter. Indeed after applying that, the report shows all "β" in the TTR column (which I've now removed because it wasn't the richt metric in the first place π )
β
β
> If so, know that the Analytics measure you're using (Findings "Time to Resolve") won't work - that measure only registers and stores a TTR value after a flaw is closed either by approved mitigation or a scanner auto-closure
β
Which metric would help me acheve the above? Have been scrolling through the available fields, but can't quite see which one is pointing to the dimension that puts the orange circles to the "more time needed"side of the "Fix First" graph.
β
β
Thanks in advance for any feedback.
Best regards,
Georges
Hi Georges,
Glad this was helpful. I'm not sure what this "Fix First" graph you refer to is, but from what you described, there are two other ways you could could consider which teams need more help or extra resources:
1) If you do have SLAs set in your policies, take a look at the Findings explore dimension "Grace Period Expiration Date." It offers the date that the flaw's SLA expires, after which point the flaw is overdue and the application will move from Conditional Pass policy state to a Did Not Pass failing policy state. You could analyze, by team, how many open flaws a team has that are very close to their due dates (however you want to define "close" - 1 week, 1 month, 1 quarter?) to get an idea of which teams are keeping up with their flaw loads and remediating comfortably before SLA and who which still have many open flaws approaching their due dates -- or even past them.
2) If you don't have SLAs in your policies, you could instead use the Findings explore dimension of "Finding Age" - it's a count of the days since the finding was first opened. You could measure the average age of open flaws for each team and see whose flaws have been open longest, with breakdowns by severity if you want to get more granular from a risk prioritization perspective.
Cheers,
Kate
Hey Kate, thank you again for the swift and detailed response! Very much appreciated.
β
We have SLA's set in the policies, but IMHO this will highlight the wrong metric as some teams are fixing the Veracode findings and others are not doing so at all. Measuring the overshoot on expiration date will put the teams on top that have made a conscious decision not to look at the Veracode findings at all anymore until more resources are provided, so that's a catch-22. I need to figure out who needs it the most for efficient resource allocation purposes.
β
That being said: the fix-first-graph I was talking about:
When going into an application and selecting "Triage Flaws" from the left menu, the top half of the screen defaults to "Source Code Viewer" mode, but you can also switch it to "Fix First Analyzer". This produces the graph I'm speaking of. I'd like a measure of summed circle area per finding criticality and prioritize budget weighted by the application's criticalities.
β
β
After some digging, I've figured out that this metric is present in the filters of the "triage flaws" module and is called "Effort" there, however this dimension's source is ambiguous.
β
In Veracode Documentation (https://docs.veracode.com/r/review_severity_exploitability) the "Effort to fix" metric is described as a property of a finding:
β
> Each finding instance receives an effort-to-fix rating based on the classification of the finding. The effort-to-fix rating is a scale from 1 to 5, as explained in this table.
β
It is the numbers in the leftmost column of that table (or at least 1-4 as they actually have an indication of effort) that I'd like to map to the max effort described in the rightmost column, which I'll have to come up with a table calculation for (if(effort==1){score += 2}; if(effort==2){score+=8}; etc.). I just found out that these calculations even exist in the analytics dashboards, so I haven't looked into how to do that yet and am also unsure if this is even possible. Worst case, I'll have to query the Veracode API and do the dashboarding in PowerBI.
β
For a score 5, I'm fine with a configurable number or leaving it out entirely; I'm comparing teams, so omitting this for all and creating a separate dashboard on the total number of 5's will give me this global idea.
β
βHowever in the analytics dashboard, effort is is a property of the underlying CWE (i.e., ${dim_cwe.remediation_effort}) and I was under the impression that this fix effort was weighted towards the complexity of the implementation. I.e., if two findings are similar but the number of Paths and number of steps is bigger, I'd assume that the latter one's effort metric would be bumped up. Maybe someone from VC can validate this assumption?
β
β
I'm aware this is a long shot, but any guidance on measuring this would be highly appregiated.
β
β
Warm regards, Georges
β