In the relative stone age of 2022, USGA rules expert Craig Winter was sitting in the living room of his Washington state home with a friend of his wife’s. The acquaintance was playing on his computer with a new application called ChatGPT, which allowed users without coding elements to interact with artificial intelligence through a simple chat text box.
When Winter finally convinced his wife’s friend to let Winter ask the super-bot a question, the subject was surely no surprise: The Rules of Golfmore specifically a question about the ease of carts. The question began and an answer emerged, with no less than the USGA’s senior director of golf rules and amateur status judging its merit.
“I was honestly surprised at how good the response was,” Winter recalled on a call with reporters Tuesday.
After more experimentation with Chat, Winter’s wheels began to turn. So did his colleagues. Could the USGA, on the back of its expertise in the rules space, team up with a technology solutions company to create an AI-powered rules engine that was better than anything any other provider could build?
The organization weighed proposals from several potential partners before settling on a pitch from one of its corporate partners — professional services firm Deloitte — that, Winter said, “showed us there was a potential that was a little bit more than we had on our plate.”
That was in April 2023.
On Wednesday, the USGA announced the fruits of that partnership: the launch of the pilot phase for Rules AI, a tool that will remain in The GHIN application and, the USGA says, gives players “an accurate and efficient way” to find answers to their rules questions.
Just mark in your request – how do yellow pins differ from red; what is the penalty for holding 15 clubs; what to do if a squirrel slips your ball, etc. — and the app will provide instructions on how to proceed in authoritative language drawn from the Rules of Golf, Decisions on the Rules of Golf, and the pool of rules questions that the USGA answers daily by phone, email, and through its rules app (more than 00 each year, each year). In total, it’s a great source of knowledge about the rules that the Rules AI has trained itself on.
Right now, the app is only available in beta mode to a select group of USGA member clubs, but the association has plans to expand that test group to about 80,000 or 90,000 club members in the New York metropolitan area later this month. The app will be available to all GHIN members by spring next year.
If you know anything about the USGA, you probably know that it treats rule governance with the diligence of an explosion physicist handling Plutonium-239. Every word is finished, every adjustment and change suffered. So it may come as a surprise that the USGA has entrusted the dictates of the game to a technology that, while extremely powerful, is also prone to delivering flawed, if not downright hallucinatory, information.
This is where the last two years come in.
“Our approach from the beginning is that we have to give accurate answers,” said Anthony Santora, the USGA’s managing director of IT, who joined Tuesday’s call with Winter. “There is a lot of misinformation out there on the Internet.” Santora added that any prospect of Rules AI providing bad intelligence was “non-negotiable. So, for us, we’ve really taken this defensively phased approach to everything.”
At first, this meant training the engine on the 25 main rules in the rulebook, along with many subsections of those rules. Then came more nuanced education via 35,000 questions and answers from the USGA’s Rules Questions Database. This was followed by feeding the bot rules questions in real time that golfers emailed to the USGA – with the USGA rules department reviewing how effectively the bot answered them. This has been an essential step in improving the intelligence of the application. Since the USGA rules have flagged AI Rules for giving incorrect or incomplete answers, they have been able to proactively coach it not to repeat the same mistake. This legal work would also make it challenging for a competitor to come in and create an equally reliable rules tool.
“We’re able to continue to make this better and be able to learn it because we’re willing, looking at it, seeing the questions coming in and seeing the answers coming out,” Santora said.
Week after week and month after month, the app got smarter and more sophisticated.
“I spent a good amount of time dreaming about the possibility of this actually working,” Winter said, “but I didn’t really get to that point until we were almost 16, 18 months.”
Each possible question is attached to one of 500 different rule topics – eg, “out of bounds”, “cart path facilitation”, etc. “When a question comes in, it will eventually come down to the topic that is most similar to what this person is asking,” Winter said. “So all of this nuance is available for the system to be able to use this closed set of information that is accurate to find the answer.”
That’s not to say that Rules AI is perfect. No machine, no matter how thoroughly trained, can be omniscient, especially when it comes to the Rules of Golf, where unprecedented scenarios can appear seemingly out of thin air. When the app receives a question it feels unqualified to answer (and no doubt, as the app becomes more widely available, an army of speedy typing rules will delight in trying to thwart the bot), it connects to refer the user to the USGA for further advice.
Complex questions can take AI Rules 10 seconds or more to answer, but more common questions will have what the USGA calls “golden answers” that the app will provide almost instantly.
Winter emphasized that the AI ​​Rules should not be used to replace the expertise or decisions of tournament committees or clubs; it’s more of a reference that allows players easier and faster access to rules instructions, which will likely also reduce the volume of rules questions on USGA courses in the old fashioned way.
“We’re not trying to put a rules official in the players’ pockets,” Winter said. “It’s about scaling the expertise of our rules staff, our rules experts, in golf around the globe.”
