Discussion about this post

User's avatar
Nan Deng's avatar

The 37% to 65% jump in 60 steps is the number I'd want broken down by choice type, because a decision model can gain accuracy by learning which options are usually wrong rather than by reading the input. A quick check: hold out one choice from training entirely and see whether the model still ranks it correctly when it appears, which is closer to how it will behave on an incident it hasn't seen before.

Abhishek Pan's avatar

The thread summary example is the one I'd test first. A session where service came back but the root cause is still open is exactly what gets flattened into "resolved" when you only retrieve related text.

No posts

Ready for more?