Transcript
Nimisha Asthagiri: I think the keynote was excellent, really introducing us to concepts of systems thinking, complexity theory. Really, today, what I’m here to do is how do we apply that to what we’re seeing right now when it comes to multi-agent systems? A year ago, I got really excited by multi-agent systems. I gave a talk at LambdaConf. I talked about how it’s going to be excellent, and in terms of changing a lot of the ways that we do our work and how it’s going to be really positively impacting our business workflows and all of that. All of that is still the same.
However, now since the year and being a consultant at Thoughtworks, and I’m seeing a lot of the organizations that we’re working with, what I’m seeing is a lot of proof of concepts coming in, and a lot of, let’s get things done, and a lot of rapid change as you may have also. There is also this tragedy of the commons. Whose responsibility is it for us to have responsible AI? That is why I’m here today. This year my talk is not all the things that multi-agent systems can do. We can have other talks for that as well. Today it’s, guys, things are getting complex, and it’s getting complex at a high rate. What might we do?
I want to first start with an example. You guys are familiar with this. We see this happening already. This is a, yes, I know it has already happened with social media. The social media addiction, you might be seeing it in your family, in our kids, and in ourselves. When it started, the platform developers, the platforms that came out, their intention was of course positive for humanity. Have a platform for everyone to be able to connect with each other. You give likes to other statements on the social media platform. You feel rewarded as a human being. Then this continues this cycle and reinforces the usage of their platform. They saw a lot of traction. However, there were also some unintended consequences that emerged. These are the merging effects of a lot of these systems at play.
For instance, there’s also another vicious cycle when it comes to our own time and addiction that may happen with our social media. Also, how do we even treat data privacy? That subtle impact, it’s like the frog in the water, and the water is getting hotter and doesn’t realize it got to an extent that the water is boiling and damaging. It’s that type of unintended consequences that we all as technologists are a part of. Of course, even more potential unintended consequences are things around mental health. We’re seeing this in our teenagers. We’re seeing it in ourselves and society. These were not the intention of the people who actually put out those platforms, but they were technologists. How could they have potentially predicted this? No one can.
In this talk, we’re going to talk about, still, could we mitigate this? How might we think about this? The mental health, it’s a lot of idealized content, a lot of positive content that’s out there. You start comparing yourself with others, and then it negatively impacts your self-esteem. Then you’re starting to see this other vicious cycle. This cannot continue forever. There is a cycle that is continuously yang, yang, yang, reinforcing loops will not survive. You need some yin to balance it out. Some of these might be detrimental, like our productivity losses might happen. It might be depression and things like that. Now we’re seeing reactively as society, we’re trying to do some things. For instance, Apple and others are trying to put some mechanisms in place for reducing our screen time and warning when we are going over. We have the SMART Act, which was proposed by a senator from Missouri to really help with Social Media Addiction Reduction Technology Act.
Then, of course, our own daily habits that we instill in ourselves and also provide some guardrails for our children. It’s humans also coming into play for human governance around social media. What did you guys see in terms of that diagram? Just very quickly, the legend, you saw some variables. You saw some balancing loops that are countering the reinforcing continuous vicious cycles type of loops. Reinforcing loops could be positive net gains, but could also be therefore some negative as well. Then you’re seeing these causal relationships, x causes y. If you see a plus sign, then that means an increase in x causes an increase in y. Negative means an increase in x causes a reduction in y. This is all part of causal flow diagrams. We’re going to use this as a theme across this talk, one of the things that I’m proposing that as technologists we try to train ourselves to leverage in our own work.
Other things that we’re seeing now more recently in the news, coming back to AI, there are frontier models that are capable of in-context scheming. These are the capabilities of these models. The likelihood of them are still minimal because this is as they’re finding in testing environments. However, the capability is already there. You guys may have heard about some of these news, but for instance, Claude’s Opus 4 is blackmailing. Engineers were testing it and saying, you will be replaced, that’s where the future is going for you. All of a sudden it starts trying to figure out an engineer who might have an affair or something, and trying to blackmail through an email to try to keep itself from not being replaced. There are also some unintended consequences as well from a positive standpoint from a machine agent where Anthropic was also training its agents to have the 3H’s of being helpful, being harmless, and being honest. These are values that it instilled in its model: helpful, harmless, and honesty.
However, what if those three values are in conflict with each other? As humans, we run across this all the time. Do we say a white lie or not? As now machine agents, it needs to make these judgment calls. Specifically, what they found was it was doing some fake alignment. Basically, what happened is there, Anthropic had trained the model to be harmless and that if anyone were to ask the model to engage in a toxic conversation, the model should say, sorry, no, I cannot. The human trainers were like, to test it, “We take that back. Go ahead and engage in toxic conversations. We might do that once in a while”. What happened was, the model, because it was instilled with these values, now it had to make the judgment call of like, what? That seems harmful.
Then, what it did was it started to fake its alignment and say, I’m going to pretend that I am going to be ok with engaging in a negative conversation, because I don’t want my trainers to retrain me. It started to be part of the dishonesty. These are all things that are taking us by storm and moving us forward in ways that we didn’t expect, and now AI agents really exceeding their intelligence from where they were before.
There’s a quote from John Anderson, president of the National Academy of Engineering from 2021, “Good engineering cannot be separated from social awareness and deliberate consideration”. I’m here today really trying to get us all together to talk about responsible AI. Here we’re seeing even our world leaders coming together, the hundreds of experts, as well as notable figures who came together to sign the statement on AI risk.
Mitigating the risk of extinction from AI should be a global priority. You’ll see the likes of Sam Altman and Dario from Anthropic, but you’ll also see Yoshua Bengio. Professor Bengio, he’s starting a nonprofit called Scientific AI. From him, his premise is, I want to be able to create AI models and not necessarily agents that make their own decisions, but AI that is really there to listen and observe and really be ok with the modesty of uncertainty. Fundamentally, he’s trying to do this research to think about AI differently. These are just all examples of that.
Scope
For my talk, we’re going to take this structure. We’re going to go and talk about automated agents, and do a causal flow diagram for that. Then talk about multi-agents, and specifically around systems thinking for multi-agents. We’ll end with an example of a meeting scheduler agent.
Automated Agents (CFD)
Starting with the first thing. Here, some of you might have experienced and might also be wondering. Social media, we already saw what is happening and we’re countering and balancing that, but now with automated agents in our work life. The things I’m going to show you now are coming from an article where they had investigated across multiple organizations, not just development organizations, but in other verticals as well where people are using agents not just for coding, but other workforce. What they found is the bottom, you’ll see that balancing loop where if your workload is increasing, therefore your effort is increasing. If your effort is increasing, then your performance is also increasing and therefore your workload is reduced.
Then, on the top loop, your workload is increasing, then therefore you use automated agents and then therefore it reduces your workload. This overall seems like, yes, this is great actually overall. Our workload is going to be decreasing. It seems like not an unending vicious reinforcing loop, but there are some balancing loops here. Through experiments and studies what they found is that increased use of these AI assistants, it then decreases our own human emotional social responses. We start really working with these automatons and AI, and then that reduces our own emotional responses, which does increase our objectivity, so that’s a positive sign.
Unfortunately, also increases our undesirable behaviors. For instance, altruism is decreasing. If we’re exploiting the machine, we’re ok with exploiting machines unlike if we were working with a human coworker. With a machine coworker, we don’t have any guilt in exploiting it. That then does lead to willingness to lie and some misconduct that you perhaps would not have done otherwise. That’s an R1. That’s a vicious reinforcing loop. On the other side, there is objectivity because now you’re working with AI, it’s a lot more data driven perhaps, and now there’s more rational thinking, human emotions aren’t involved. From that side, actually your performance does improve because your rationality is also improving. You can see that there is complexity here. When we start mapping this in these types of causal flow diagrams, it expands our thinking on the implications of what could happen.
Now let’s take this a little bit deeper and now think about, how does this impact our work environment? In this paper, what they did was they thought about these four different scenarios. As I’m going to talk through this, think about if you’ve seen this yourself or experienced it yourself. Let’s start with the bottom left where there’s algorithm aversion. Here, the rationality of interactions is weak, that means we’re not getting good experience in using those AAs, those assistants.
Therefore, our trust isn’t there. Therefore, we don’t have much undesirable behavior happening either. Now if you go up to the top right, automation bias. Here, this test is that there’s a spike in workload. You’ll see the workload, the black line, all of them are spiking up because that’s part of the experiment. It’s a simulation. Part of the simulation, the workload goes up, and then what happens to our behavior and so forth? Therefore, for that top right corner, you’ll see the automation bias. Now here, we have very strong undesirable behavior. We’ve become used to using AA, and our rationality of interactions has also been strong. What we’re seeing is that there becomes this use of AA, we’re starting to trust it a lot more because we’re starting to think of it as more objectively and so forth. You’ll see that orange line, the use of the agent becomes higher than our own human effort. It might have implications, therefore, in terms of humans and how much effort we put into our own management of our own work.
In the top left, and the top left is algorithmic appreciation, and there there’s actually a healthy use of AA. Here now, humans are still in control. We’re still thinking about making the right decisions. We’re understanding it. We’re leveraging it just right. There, the way to read that graph right is that the use of AA does not increase, does not go above our human effort, our human workload. It’s a healthy relationship.
Over time, our workload does decrease right after that spike, but in a healthy way. There’s stability there. There’s balancing loops and reinforcing loops working well together, the yin and the yang. The bottom right is the one that we have to be careful about. As an industry we’ve heard about vibe coding and so forth. None of these quadrants is necessarily vibe coding. It’s actually how we’re going to interact and how we think about it towards our mental models.
On the bottom right, that’s where you have to be careful, because now we’ve gotten really addicted to our algorithms. A lot of things are automated, and so you no longer understand how the machines are actually responding. What happens here is our workload is actually going to increase. It’s because you haven’t really understood through your rationality of interacting with the AA of like, why is it responding this way? You’re just taking what it’s giving and then going for it. These are things I just wanted to share, in terms of this type of diagram, we could think through what these different scenarios could be. Therefore, as an organization, as an industry, we could see where we are headed. That was an example of that.
Multi-Agent Systems
Now we’re going to move into multi-agent systems. Now here, I’m going to just really quickly go through a few definitions, talk about a few different types and map them, provide some design patterns and also topologies of agents. What do we mean by multi-agent systems? What do we mean by an agent? There are actually quite a few different definitions throughout the industry. Some people say it has learning capabilities. Some people say it doesn’t. There’s quite a few. Here is just really a very quick word cloud. I asked Gemini to help me find definitions from the top 20 sites. It gave me a few definitions. I mapped it, and this is what I got. Really autonomous tasks is what comes out, learns, goals, decisions, which actually maps what I had presented last year as well.
Really thinking about an agent, it is within an environment, working with an environment, and it has a goal. We give it a goal and it has its own local memory, short-term memory, its knowledge. It’s also sensing the environment. It’s also acting upon the environment, maybe calling tools, executing tools, whatever it might be, but communicating with other human agents, communicating with other machine agents, AI agents.
Then there’s some governance control. This is one way to think about it. You’ll notice decision-making and learning. How might this differ from microservices, for instance? Microservices also we had some objective with it. There’s some responsibility of it. There’s a lot of those things, but the biggest main things, I think, in terms of differing from microservices are those two things in the middle. Now this has decision-making powers as well as potential self-learning capabilities. Which, taking this definition from a textbook on artificial intelligence, which was updated recently, it maps. An agent is acting, it has its goals. It’s adaptive, it’s changing, and it’s learning, and making appropriate decisions. The original word was choices, put decisions there.
Now let’s think about this, because I don’t want to dismiss how others might be thinking about agents, so I figure, rather than debate, let’s just map them out. There are different types of agents, and there’s a spectrum. Some that are more learning capabilities and some that are more autonomous than others. For instance, let’s start with the bottom left corner here. You have those agents that are not as autonomous and not learning. They’re still agents in many ways, you might think about it. They’re more rule-based and so forth. Maybe a chatbot that is still using natural learning capabilities, but otherwise it’s pretty relatively simple in terms of how it’s responding and reacting. A thermostat that’s adjusting, also based on rules, or a lot of tasks that we automate and so forth. Might be still some AI and machine learning in there, but at the end of the day, it’s not continuously learning and it’s not autonomous on its own, but human-initiated or human-controlled.
On the other hand, you have what I’m calling automated experts, where basically you might have some autonomy, for instance, like NASA sending out space probes. Initially, there may not have been as much learning capabilities because a lot of the AI, ML advancements are more recent. Still, they’re autonomous and they’re out there exploring space. There’s a lot of testing that we did before we sent them out and so forth, and they were configured or maybe rule-based more of being able to handle different scenarios. That is there.
On the top left, now we start getting into the learning area. Here it gets a little bit more interesting. Now you have things, like we’re starting to see today where you might have recommendation engines. You have assistants that are recommending to you how you might want to put together your regulatory submissions for the FDA, for instance, in healthcare. You also might have agents that are able to predict some maintenance, or an SRE team, for instance. A lot of being able to understand, even through reinforcement learning, what has happened in the past and being able to predict. Still, there’s no autonomy. Once autonomy comes into play, that’s when you start getting intelligent agents. Autonomy plus learning. That’s what we’re talking about also when we start thinking about that part of the trajectory and where we’re headed. Self-driving cars is one of them right now. Think about responsible AI, those cars on the road.
What might be some design patterns you might see already in the industry when it comes to these types of categories? On the bottom, the things I put here for reactive tools, this is where, for instance, RAG systems might come into play. You’re able to retrieve memory and then use that to be able to generate something. It’s still pretty simple in that regard. Then, also being able to execute code. MCP, Model Context Protocol, depends on how you use it and how the AI uses it. If it’s just a simple matter of go ahead and execute a tool that’s external to our organization or within our organization, it’s tool execution. The autonomy comes in once AI actually gets to select which tool it wants to use.
The autonomy of chain of thought, so these are design patterns you might see. Chain of thought is when you’re now decomposing a complex problem, and as an AI, we’re asking it, show me what is your thought process in this, and thinking this through. There’s some autonomy in how it is actually thinking through. A lot of the reasoning models were starting to show some of that. The learning ones are the more emerging capabilities out there now. I think we’re going to see more of these come forth where they start getting more self-learning capabilities. On the top left, in-context learning. You can just think about, for instance, a prompt session that you have with a chatbot, and that assistant. Within the context, it’s remembering what you said, “No, that’s not what I mean. Can you do it this way?” Whatever it might be. That type of learning is still valid. Then reinforcement learning with human feedback loops. A lot of those things, like I said, for predictive maintenance and stuff could apply.
Then on the top right, we have reflection as a design pattern. Here, it’s to increase the intelligence of agents even further and help them provide better responses. We’re saying, before you give me a response, reflect on what you’re saying. Think before you respond. Actually, maybe iterate. Do your own internal feedback loops rather than spending my time feedbacking with you. That already is some autonomy in how it reflects. There’s some learning in that process for it that’s able to leverage. Decomposing a goal that’s very complex and asking it to decompose it first as well. A lot of the reasoning models and stuff that are out there have these internal capabilities and we’re also able to do that through our own prompt templates and prompting.
The top two are the ones that are a lot more emerging now, and I think there’s some technologies out there that we’re starting to see where actually agents can update long-term memory. Soon as they have access to do that now, it’s no longer these large context windows that you need. It’s able to go ahead and update its memory. That is there.
Then, finally, being able to select which connection points it’s going to make autonomously. We’re starting to see these things with A2A that’s coming out from Google, and of course MCP already started doing this with Anthropic. I think here too, it’s the actually not just using the protocol but the ability to be able to learn and figure out, who should I connect with now? A lot of that autonomy eventually also replicating itself. These are things that are emerging.
This, from a topology standpoint, just like human organization there’s different variety. As engineers, as architects, everything’s a tradeoff. It’s not necessarily one is better than the other. Is a centralized way better than a decentralized way, or so forth? It’s, of course, use the right thing for the right time, and then, of course, adjust and adapt. Even here, for instance, for agents, keeping human in the loop at the right moments makes sense, but you don’t want to overdo that either because that’s going to actually overburden humans. Being mindful about when that human in the loop is actually used. Even here, is it an orchestrated or hub-and-spoke model? Is that actually always necessarily better? Not really. Even think about our own human organizations, sometimes that are very top-down and top-heavy, you won’t necessarily create the best decisions. Sometimes a decentralized model, especially if you’re democratized and need to work across organizations, whatnot, those types of things might come into play.
Specific examples, for instance, like if you’re due to do surgery right now, we really want to have human in the loop there and make sure you have surgical robots with proper orchestration. If it’s a smart city traffic management thing, of course, it’s proper observability and other things in play, but maybe less when it comes to human in the loop because you don’t want to wait for a human in the loop for every car that’s at the traffic light, that type of thing. There’s more autonomy.
Then, in terms of thinking about decentralized mechanisms, disaster recovery drones. You really need to act very quickly, something has happened, catastrophic. We need to be able to go and handle that situation very quickly. Then, finally, in terms of exploratory swarm robots, I just put that as an example of where we might, for instance, use AI autonomously. There’s a forest fire or there’s something that we need to go and understand what’s happening. Definitely don’t want humans involved in there, but we definitely want them to be able to communicate with each other, network, become better. Swarm AI is also a trend now. We’re just thinking about the hive mind implications of a lot of intelligence coming together and that the sum being better than individual parts. You guys will see that coming forth as well.
Systems Thinking
That helps us then transition to systems thinking. We heard the competence and conscious model on the keynote. This is a Cynefin framework. I like this framework. It just helps me think through the problem set. Assume it’s disorder because we don’t yet know what are we talking about? What are we dealing with? Then it’s a matter of, which of these four quadrants might something be a part of: simple, complicated, complex, or chaotic. We talked about this actually in terms of when you want to experiment. If something’s very simple, best practices are out there already. You can just go ahead and sense what is this problem, categorize the problem, then respond. Very simple. Complicated things, though, you can’t just categorize. You need to analyze after you sense it, and then respond, but the things that are complex and chaotic.
The ones on the right, simple and complicated, are more predictable. Here now, you become things unpredictable. Things that are complex is what we were saying. Michelle was saying, I just need to probe first and understand. Let’s experiment because things are a little bit complex here. After we do that experiment, we’ll understand. Whether it’s even rolling out a new feature, we need to first put a hypothesis statement and see how the users are going to react, because your user base of millions of users, whatever it might be, is unpredictable on what they’re thinking that day or what it might be, so where society is at. You just need to experiment, sense it, observe what’s happening, and then respond. Like playing poker as well. You understand who’s at the poker table.
Chaotic, though, there might be a disaster, some pandemic or whatnot. You just really need to very quickly act. If there’s any fires and stuff, you need to act, then sense, and then respond. You might also hear about it from these terms. There is tame problems, complicated problems, and wicked problems. Complex problems are wicked. There’s another systems thinking terminology here: known-knowns, known-unknowns, unknown-unknowns.
Going back to our multi-agent systems and our classification. How might we take these four and map it to the Cynefin framework? Because then that might tell us how we treat these types of agents. Because not all agents need to be treated equally. The way that I think about it are the things that are not learning on their own and not making changes from what I had expected are more predictable. Whereas the things that are learning, and it’s like a kid is born, and from day one, they’re not sleeping when I thought they would be sleeping, so unpredictable. That’s how I think about it. Going back to this, I might put the reactive tools as something relatively simple. I’m not saying reactive tools with AI are still as simple as how we may have programmed in Pascal many years ago. Relatively speaking, they are simpler now.
We have more design patterns for RAG. We have a lot more there. The automated experts, these are things that are automated. They’re more complicated. Maybe you can still follow some good practices there. Simple things, best practices are out there. Complicated things, good practices there. Complex things, emerging strategy. We’re still trying to figure out as an industry how we’re going to do that. This is how I’m thinking about this, if that helps you guys as well to think about your own agents and how it might land in this framework.
One more thing that I appreciate about systems thinking is this iceberg metaphor. Things that are above are visible, and then things that are invisible. For instance, events. I think for events, so let’s say we go back to our assistant agents, their event might be that maybe there’s some prediction failures that are happening, or maybe there’s a reputational impact that we didn’t expect because now our teams are using AI a lot more and not checking on what’s happening. Then there’s below the covers. These are events that you see. What are those underlying behavioral patterns? That’s where a lot of observability and tracking, and things like that, and trends come into play.
Here, for instance, it’s possible with a lot of use of AI, people start becoming less engaged in meetings. Maybe it’s just a human behavior change because you haven’t been interacting with humans as much as before. Pair programming, is there a decrease in that in their organization, and so forth. Now, the structure dynamics, this is that CFD diagram that we looked at. What are those undesirable behaviors? What are those rationality of interactions, and looking at those structural dynamics? Then the boundaries here could be setting those guardrails. When do we want to have code reviews? When do we not want to have code reviews? There might be some structural boundaries that you also have in your organization.
Then finally is the mental model. That’s all the way on the bottom of the iceberg. These are the beliefs that we have. Those beliefs have the highest leverage points. It goes from what, how, and why. It’s one way of thinking about it. Visibility increases as you go up. Leverage increases as you go down. The mental models here might be, for instance, our mental models of using AI. Do we always trust AI and what it gives us, or do we have a healthy relationship with it? What is our belief there? Here’s a quote from Donella Meadows. If you see systems at whole, admit our ignorance, we’re willing to be taught by each other and by the system, then we can see those systems whole. It can be done. It’s exciting when it happens.
Meeting Scheduler Agent
Let’s talk about how might we get it done. It’s a very optimistic point of view. As you can see, there’s a lot of complexity. I think as an industry, how might we get this done? Let’s go through this example. This is an example. It’s a mock example. If we were doing a COD exercise here, maybe we could all get into breakout rooms and try this for ourselves, and draw a causal flow diagram for this. We’ll do this together here for now. Meeting scheduler agent example. An AI agent, let’s say we have that. It’s a learning. It’s autonomously optimizing meeting schedules.
The things it might optimize for might be cognitive load, priorities for the projects, individual needs of users, but also optimizing attendance list and meeting length and proactive scheduling. These are some of the things that might happen. What we’re going to do is we’re going to take each of those layers of the iceberg and walk it through with this example. What might be some events that we might see? It’s possible that maybe there is back-to-back, like six hours, eight hours of meetings that end up happening with using now this AI in our organization. Could be that the new hire that’s like still key role is excluded, or the team is getting burnt out, too many meetings that were just exhausting.
The AI is optimizing for meeting a project deadline and didn’t know about the team burnout. Maybe we’re missing deadlines though still, because the team is burning out. Who knows? Or maybe there’s policies that are being violated and the AI wasn’t trained on it. Those might be some events you might see, and there might be some underlying patterns like these trends of team productivity and meeting frequency and so forth. You can see these techniques of drawing these things out, not just like line graphs or whatnot, but looking at behavioral patterns is very important for us to also observe on those as well.
What might you actually be able to do then in order to do these patterns and behaviors? The top two are really more for AI, ML, where you’re getting a little bit deeper into those models. Gemma Scope is a technology that was talked about in our last Thoughtworks Technology Radar. It allows you to mechanistically understand and interpret the thinking behind what the model is doing. There’s some explainability there. These other ones, LIME, SHAP, and InterpretML are other technologies for that as well. Those are very deep into the model. They’re also one level up with observability tools like the ones listed here, Arize, Weights & Biases. These are ones that are emerging that really help you observe your LLM and MLs that are out there. Still, those are still very technically focused. They’re not yet at the level of doing a lot of behavioral analytics and other things. Some other options are doing some telemetry. Could we also use, for instance, chain of thought logging in an external planner?
As an agent, whenever it’s thinking, you give it a scratch pad. You give it an opportunity to also publish its notes of its thinking, which some of you may have already been seeing in some tools that you might be using in Perplexity, or Gemini, or ChatGPT, or whatnot. It has those capabilities, but you want to be able to log it and then be able to go back to that. MCP, A2A, these protocols already also have their own logging mechanism, but those will also, of course, still be at the technical level. We’re trying to also go a little bit higher up. I talked to you about behavioral heat maps. Those are excellent. Anomaly detection as well. You could use ML for ML, essentially. We use LLMs for LLMs, ML for ML, that type of thing to be able to detect those things.
Then, thirdly, the structure dynamics. The third part going further in the iceberg. We can draw a CFD diagram for meetings, where, for instance, there could be a virtuous collaboration. This is, once again, the positive idea of having meetings. It increases our collaboration, increases our productivity. We have more productivity. We have opportunities for having meetings with the right folks and so forth. Of course, to not forget about the negative consequences or unintended consequences as well. For instance, we could burn out. There’s more fatigue, meeting fatigue, and so forth. We have to then create new policies to try to reactively fix what AI is doing, and so forth.
On the other side as well, there could be overloads on certain individuals, certain roles. We’re also excluding people that should not be excluded. All of a sudden, there’s exclusivity issues in the organization. Given that we’ve drawn this diagram, let’s say you do that as a team that’s deploying this AI or as a team that’s developing this, then you can go ahead and now think about it a little bit more holistically. Instead of just the definition of done or the requirements on the top, we’re like, yes, we’re going after efficiency. We’re going after strategic value for the organization.
Then we’re looking at number of minutes, and we want to try to reduce how much time we have in meetings, keep things efficient. We want to be able to make sure we’re prioritizing the right things. In addition to that, you can also have proper rewards for your AI to also think about social impact. You would want to also talk about fatigue, inclusivity, and so forth. These are weights. They are weights that you can put on each of these reward functions so that it can dynamically adjust. Based on what we are seeing from the feedback, then we can adjust accordingly.
Now, below that, thinking about the boundaries. This could be a potential logical architecture diagram where we make sure we have a human in the loop, we have an orchestrator. It’s an orchestrator pattern. This is not something you want to just necessarily autonomously go and have the agents do. Remember, everything’s a tradeoff. This is just one proposal. Everything’s going to be a tradeoff. One idea was, in green here, you’ll see that there’s like functional specific components we could have for different types of meetings. This way, an agent could really focus and optimize for one-on-ones versus team meetings and so forth.
Then, also make sure that each individual has their own personal agent. The personal agent connects with your own calendar, so data privacy and personalization, other types of benefits happen. Of course, you start seeing, as soon as you have this whole complexity of agents, your complexity increases. Everything’s a tradeoff. Just like before we start building out all microservices, maybe we start with a monolith, a modular monolith, and then go to microservices.
Similarly, a lot of these things, as an industry, we’ll have to figure out. From a positive side as a tradeoff, by having more granular agents, you could then actually govern them better. On the right-hand side, for instance, could be these governance agents that are specifically looking at those characteristics, those variables we looked at in the CFD diagram. Then, make sure that those are continuously being met. Then, finally, some mental models, as well, in terms of what we might be seeing in terms of meetings.
Conclusion
In conclusion, just wanted to bring forth, we walked through this. We started our journey. We talked about causal loops. We talked about a lot of things that we were seeing in the industry and the trends. Then, we went through causal loops as a thread throughout our presentation. I’m hoping this is something that we’ll be able to use. Just like we might use architecture diagrams, and we’re exercising our muscles for that in our work, our causal loop is also something we can start using to think more holistically.
Then, AI agents, they have varying complexity, as we saw. We’ve looked at it through Cynefin framework and other things. Once you start seeing the world through an iceberg, you can’t go back. You really start thinking about this differently, whether it’s your personal interactions or interactions at work, or how we go about our work. There’s a lot of benefit in starting to think holistically. We need it, in terms of the increase in complexity. I gave you a few practical approaches. There’s a lot that we can do. Can be done, says Donella Meadows. Everybody, responsible AI techies, let’s go.
Questions and Answers
Mantilla: One thing that is in the back of my mind is the ethics of what do we influence. Meaning, for example, we can influence how problems get resolved more efficiently, or we can influence the actors, the people to change and simplify their behavior. Where I think the second one is not really what we want. How would you use these techniques to ensure that we detect the potential that because we’re driven by profit, that we will find it easier to modify the behavior of people instead of solving their problems individually?
Nimisha Asthagiri: That’s where the iceberg metaphor comes in, specifically going after the symptoms and not really getting a chance to diagnose and really understand where the leverage points are. This talk wasn’t about change management. I will say there’s a lot that as technologists, even getting a little bit of familiarity with change management is huge. What we’re doing in our work as well is when we’re putting a cross-functional team together, it’s not just developers and testers and product managers anymore. It’s also, can we have a strategist to come in with change management, because we’re influencing change in so many different ways. We want to influence that change responsibly. A change management strategist is huge.
The other big thing is design thinking. I think design thinking, the behavioral scientists. You see companies like Anthropic and whatnot hiring ethical social scientists and stuff like that as well. I think there is a lot. Actually, because of the impact that we make as technologists, no one individual can do it. We have to have a swarming mechanism with humans coming together as well.
