We recently had the great pleasure of hosting Brooke Carter at TechKnowCon.
She is a Program Manager of Machine Learning Education at Uber and the author of Profreshional: A Guide to Surviving The Freshman Year of your Professional Life.
Her presentation showed attendees how education can help scale machine learning, and why applying engineering principles to program design can be valuable for tech learning.
Here are a few more specific takeaways.
Uber sees machine learning as an integral part of its operations
Brooke opened her presentation by noting that the company’s CEO, Dara Khosrowshahi, once described the company as “a giant machine intelligence problem.”
“I really like this quote,” said Brooke, “because I think, at a very high level, it does a good job summarizing why the company would think investing in formal education within the machine learning domain is important. We have millions of productions being made every single day. And we have quite a wide variety of employees who work on implementing machine learning solutions.
Uber needs a very well-informed and very confident and empowered workforce who feels that they have the tools that they need and can implement machine learning solutions and to any project or program they’re working on that might require it.
– Brooke Carter | Program Manager (Machine Learning Education), Uber
Carter shared, “our response to filling that need was establishing a formal education program.”
The 4 key areas they chose to improve
Uber focused on solving problems with knowledge distribution, model quality, user onboarding and fragmented resources. These four areas presented numerous challenges as Uber went about creating a system capable of distributing the unique knowledge held by the company’s machine learning experts. Fortunately, they were able to tap into a talented cross section of employees.
“Right after obtaining buy-in, we assembled a strategic cross-functional steering committee,” said Brooke. “This group is referred to as the Machine Learning Core Team, and they remain a fantastic resource to this day. They helped aid in a lot of strategic decision making, especially early on in the program’s life cycle. We have subject matter experts from other tech learning pockets. We have machine learning engineers. We have directors from our Uber AI organization that sit on this committee.”
Validation came first
Having designed a formal education program around machine learning, Uber first tested it before opening it up too widely.
Before anything else, we launched a few inaugural courses just to validate this concept of a formal education program and validate the frameworks that we had built.
– Brooke Carter | Program Manager (Machine Learning Education), Uber
She expanded, “the group working on our internal machine learning development platform was a sort of haven of beta testers. We rolled out the first few offerings to that group, got some feedback, and we were able to ensure that our development process for content was solid. Next, we went out more broadly, advertising through many other channels and widening our participant pool.”
Engineering principles informed the design of machine learning course flows
This seemed to be a particular point of pride in the presentation.
“We have a five-step process that takes roughly two and a half months to build a full-length course offering,” said Brooke. “And what I think is the coolest part of the development cycle is the engineering principles that are embedded within every step of the process. We were inspired to apply engineering principles to the course development design and the broader program design, knowing that a subject matter as complex as machine learning could do well if it had these principles applied at the forefront. Embedding capabilities like reproducibility into our course design from the onset of our program has helped with the overall effectiveness of the learning offerings that we provide to users.”
The 3 types of modalities they used
This curriculum was designed to cater to a variety of learning styles, mixing in several modalities.
“A course can be delivered in three different ways: live, semi-guided or online,” said Brooke. “An online live course means that we have instructor resources present on zoom, walking through the different modules and helping troubleshoot via Slack. Semi-guided is a blended learning approach. This seems to work well for low complexity theory content, where a user can be tasked with just reading the high-level concepts on their own before jumping into the group hands-on portion. Finally, online learning is an offline version of the live courses. It’s exactly reproducible to what is seen in real time.”
Automating the tedious stuff frees up headspace for deeper work
“We use tools and templates to streamline things like course initiation and course planning,” said Brooke. “We use tools to track our course development and make sure that everyone is on the same page and held accountable for deliverables.
We have a learning management system that we’ve used to automate the course enrollment process, which has been a lifesaver compared to our previous, very manual process. It helps by spreading the word about course offerings, gauging interests, and signing people up. Now we’ve freed up so much more time to get to the fun stuff, experimenting with things like office hours and lead initiatives.”
– Brooke Carter | Program Manager (Machine Learning Education), Uber
The classics metrics are classic for a reason
When gauging the success of its programing, Uber often relies on tried-and-true measurements.
“We have a 93% aggregate OSAT across all of the live courses that have been facilitated in 2022,” said Brooke. “We’ve had over 600 attendees for our live offerings, and we have over 2000 unique views of our co-op activities, which are the hands-on modules that are walking individuals through practice scenarios that they might encounter in production.”
Brooke ended her presentation by stressing just how much she loves her job, which, frankly, didn’t need saying. It was clear from the outset that she’s incredibly passionate about machine learning and course design, and we really urge you to go back and watch the full video of her talk. Contained within it is a blueprint for how to create successful programs for technical learning.
