As an AI team manager, Vivek Gupta stays broadly informed to guide AI experts effectively and drive the team. Engineers need feedback on both technical and interpersonal skills, Gupta mentioned in his talk Growing and Cultivating Strong Machine Learning Engineers at Dev Summit Boston. He stresses learning time, asking for help, and cross-team collaboration. Mentorship, data handling, and human-in-the-loop validation are key to success for machine learning engineers.
As a manager of AI teams, he has to know a little bit of everything, Gupta said. He needs to be familiar with the applied sciences, at least to understand where the value lies, and he must stay current. His senior engineers are the experts who will dive deeply, but he needs to have ideas to help drive the team forward, he added.
One of the primary things engineers are seeking is feedback. They’ve just come out of school and are used to getting grades, and they want to know how they can do better, Gupta explained:
Feedback is a very varied thing. Some of it will be about how they are doing in their coding areas, some of it is actually on how to interact with others, or how to deal with other people and teams that they’re working with.
To nourish engineers, give them time to learn and try out new things and practice with it, Gupta said.
Gupta mentioned that we need to get engineers to ask questions. Typically, they’re not asking questions until they’ve really been stuck for much too long, he added. We need to actually encourage them to go to the senior engineers and managers, and ask if they know somebody who might be able to unblock them.
We want people to talk to other disciplines, talk to people on other projects in order to foster collaboration, Gupta explained:
Often, there are ideas on other teams that may allow them to leverage work that somebody else has done or share something that they’ve done to reduce duplicate effort. Encourage that type of collaboration or listening in on other people’s talks or project design presentations, so that they can learn from that.
Senior engineers can be mentors for juniors. Coaching seniors on how to do mentorship can make it more scalable for the organization, Gupta said.
People working on machine learning in a production environment need to understand how AI and machine learning are done by data scientists. Gupta mentioned that they also need to know about data management for machine learning, which is different for machine learning:
You have to keep track of what data is used to train your models, and of test sets that you might use to validate your models. You have to move data from one place to another, maybe reformat lt, or do aggregations.
Consistency in how you manage your data for training is important. Gupta suggested automating for frequent retraining by building training pipelines.
We need the human in the loop to validate answers, check code that was generated, or compare different alternatives. User feedback is what closes the loop, Gupta said. Thumbs up, thumbs down isn’t just about how good a job you’re doing, but it is giving you feedback on how your model or models are performing, and which ones may need some modification, he concluded.
InfoQ interviewed Vivek Gupta about growing machine learning engineers.
InfoQ: What do you do to enable engineers to learn new things and try them out?
Vivek Gupta: We regularly host hackathons on our team and participate in the Microsoft-wide hackathon annually. In addition, we have a day of learning at the end of every sprint (2-week sprints for us). Our team also has lunch and learn sessions to share our learnings or to bring in guest speakers. Lately, much of the learning has been around agents and using AI assistance for coding. These areas that they are each developing experience in so each has an opportunity to demonstrate something new they have learned.
Although much of our learning focuses on tech, there is another side to their learning and development. We give them opportunities to learn about managing their career, what managers/tech leads do, and how to evaluate their impact. This often comes in the form of bringing in more senior speakers, prior cohort members, hands-on opportunities with interns, and sometimes supporting another team by acting as a PR reviewer/tech advisor.
InfoQ: What does collaboration look like for senior engineers in your team?
Gupta: For our more senior engineers, collaboration is about learning what is going on across teams, helping review PRs, participating in design reviews for each project, and helping lead learning sessions for new team members. This leads to fostering knowledge sharing and also setting people up as natural technical leads across our team. Often, it feels easier for junior team members to go to senior engineers vs going to managers.
InfoQ: How can MLOps help to properly manage large language models?
Gupta: We have some of the same issues with large language models as we did with more traditional models. We are now fine-tuning them, so we need to keep track of the data we used to fine-tune them. We need pipelines for evaluating our prompts, and we need to keep a library of prompts for different models. Although LLMs operate differently from what we did in the past, the learnings for MLOps still apply to making sure our approach is a well-engineered approach for production scenarios with LLMs.
