I hope it gets better. Just for fun I’m posting the experiment I ran in my last newsletter. I wanted to see how good it was if asked good questions around Scrum (the focus of my work). I will let you judge the results:
I think many of my readers have tried ChatGPT or in the past few weeks. I thought it might be fun to see how good a job it does.
Describe the purpose of Daily Scrum
The Daily Scrum is a short meeting held every day by a Scrum team to synchronize activities and create a plan for the next 24 hours. The purpose of the Daily Scrum is to inspect progress toward the Sprint Goal and to identify and remove any impediments that are blocking the team from meeting that goal. The Daily Scrum also helps the team become more efficient and productive by ensuring that everyone is on the same page and has a clear understanding of the tasks they need to complete. Additionally, the Daily Scrum helps to foster collaboration and communication among team members.
On this one it does an excellent job. It is missing all of the mechanics - but I didn’t ask for that. It misses the idea that the event was intended to eliminate other meetings from the team member’s work day.
The tool should take a bow, it does better than most humans so far. Letter Grade: A
On to the ScrumMaster:
Describe the role of the ScrumMaster
The ScrumMaster is responsible for facilitating the Scrum process, ensuring the team is following the Scrum framework, and removing any impediments the team may encounter. The ScrumMaster also serves as a coach and a mentor to the team, helping them to become more effective and efficient. The ScrumMaster also helps the team to identify and prioritize tasks, as well as manage and monitor progress. The ScrumMaster is ultimately responsible for ensuring that the team is successful and that the Scrum process is adopted and followed.
This one has some more issues:
- “removing any impediments the team may encounter” - True - however it doesn’t cover the oft missed point, much of that comes from coaching the team in how to remove their own impediments
- “The ScrumMaster also helps the team to identify and prioritize tasks, as well as manage and monitor progress.” - The SM has no role identifying tasks, prioritizing them. Nor do they manage and monitor progress.
Outright missing (quotes from the ScrumGuide): “Coaching the team members in self-management and cross-functionality”, “Helping the Scrum Team focus on creating high-value Increments that meet the Definition of Done” and “Ensuring that all Scrum events take place and are positive, productive, and kept within the timebox.”
So far on ScrumMaster they might get a C- for turning the ScrumMaster into a manager and missing so many elements from Scrum the ScrumGuide
Finally this answer completely misses their service to the Product Owner and the Organization. So I asked it a supplementary question:
Describe the role of the ScrumMaster in coaching the Product Owner
The ScrumMaster serves an essential role in coaching the Product Owner. The ScrumMaster works closely with the Product Owner to ensure that the product vision is clear, the product backlog is well-prioritized, and that user stories are well defined. The ScrumMaster also helps the Product Owner to understand the value of each story, and to ensure that all stakeholders are working together to move the product forward. The ScrumMaster also works to ensure that the Product Owner is taking an active role in the product development process, and that they are making decisions that are in line with the product vision. The ScrumMaster is an important source of guidance and support for the Product Owner, and helps them become a better leader and product owner.
Scrum doesn’t require User Stories, but let’s ignore that detail.
Issues:
- “to ensure that all stakeholders are working together to move the product forward.” - Scrum doesn’t require the stakeholders to work together, I’ve encountered more than few situations where the stakeholders represent competing factions in an organization.
- “… works to ensure that the Product Owner is taking an active role in the product development process, and that they are making decisions that are in line with the product vision.” Yes the PO needs to do their job, no the ScrumMaster isn’t their baby sitter.
- “…an important source of guidance and support for the Product Owner, and helps them become a better leader and product owner” - More of the same, my late father would have called this drivel
Outright missing (quotes from the ScrumGuide): “Helping the Scrum Team understand the need for clear and concise Product Backlog items” and “Helping establish empirical product planning for a complex environment”
Ok I’m giving it a D - here, it’s barely passing.
So when it comes to explaining basics of Scrum our machine learning tool is currently getting a C
Last thing I thought I might try ask a real coaching question from a recent Lean Coffee:
What functions/tasks should a BA perform on a cross-skilled scrum development team?
- Facilitate communication between the Agile cross-skilled development team and stakeholders.
- Develop user stories and use cases to ensure customer needs are met.
- Analyze customer requirements and create a product vision.
- Participate in sprint planning, daily stand-ups, and retrospectives.
- Prioritize tasks and manage the product backlog.
- Identify and remove roadblocks for the team.
- Help break down complex tasks into smaller, achievable goals.
- Monitor progress and report to stakeholders.
- Develop and maintain a product roadmap.
- Make sure the product meets customer goals and expectations.
Wow. At best they turned our BA into a warmed over product owner. This answer is so sad, I will just grade it an F and go off to make coffee. Do you want to discuss a better answer without any AI? Come to our next LeanCoffee.
Apparently the AI needs some more real world work experience before we set it loose on unsuspecting victims. To be fair, I suspect the errors are example of what is said in the training data. The old adage Garbage In, Garbage Out predates machine learning by over 60 years.
The funny thing, my father complained AI was promised his entire career and never got it. I can tell him we’re still waiting.
[1] GIGO Garbage in, garbage out - Wikipedia - “The first known use is in a 1957 syndicated newspaper article about US Army mathematicians and their work with early computers, in which an Army Specialist named William D. Mellin explained that computers cannot think for themselves, and that “sloppily programmed” inputs inevitably lead to incorrect outputs.” - Article clipped from The Times - Newspapers.com™
