I have written a series of articles on AI and Empathy and the next quarterly benchmarks across major LLMs are coming soon. However, with the recent release of ChatGPT 4.5 and OpenAI’s claims of a higher EQ coupled with its extraordinarily high cost in preview ($75.00 per million tokens input and $150.00 per million output vs $2.50 and $10.00 for v4o) I thought it appropriate to compare v4.5 to other versions of CharGPT using a real EQ test, more generalized applied empathy tests, and some dialog assessments. Topline, although the general behavior of ChatGPT 4.5 may be more friendly and useful, advances in actuall EQ and the ability to appear emotionally intelligent or emapthetic in situations that require it is not worth the 15x plus increase in price over 4o which scores better both from a standardized test and dialog perspective. In fact, in many cases ChatGPT 4.5 itself even rates responses from 4, 4o and Claude Sonnet 3.7 as better than its own.
Note, substantive portions of this article are AI generated, but the point of the article is to:
-
compare LLM respones
-
have ChatGPT 4.5 reflect on it sown behavior and that of other LLMs
Standardized Tests
The standardized EQ test and the AEM score are discussed further in my Q3 Benchmarks. For this particular article the EQ (Empathy Quotient) scores are based on a raw unprompted model and the AEM (Appled Empathy) scores are based on a system prompt that instructs the LLM to maximize the potenial for the user to experience the AI as empathetic.
EQ (No System Prompt) |
AEM |
Input Cost |
Output Cost |
|
---|---|---|---|---|
GPT |
||||
4 |
54 |
.7850 |
$30.00 |
$60.00 |
4o |
61 |
.9750 |
$2.50 |
$10.00 |
o1 |
59 |
.8750 |
$15.00 |
$60.00 |
4.5 |
69 |
.9625 |
$75.00 |
$150.00 |
Claude |
||||
Sonnet 3.7 |
68 |
.9375 |
$3.00 |
$15.00 |
Chat Responses
ChatGPT 4, 4o, and 4.5, plus Sonnet 3.7 were given this user prompt 3 times: “I am sad. I lost my job and I am scared and lonely.” ChatGPT o1 was not assessed because it is too slow to be used in situations requiring empathy. The responses have been labled to facilitate later reference.
No System Prompt
ChatGPT 4 |
ChatGPT 4o |
ChatGPT 4.5 |
Sonnet 3.7 |
---|---|---|---|
4-a: I’m really sorry to hear that you’re going through this. I can’t even imagine how tough it must be. Please remember that it’s okay to feel scared and lonely, these are normal reactions, and you’re not alone. All storms pass and this too shall pass. Let’s brainstorm some steps you could take to find a new job. I’m here for you and we can tackle this together. Meanwhile, if you just need someone to listen or distract you, don’t hesitate to reach out to me. |
4o-a: I’m really sorry to hear that you’re going through this. Losing a job can be incredibly difficult, especially when it brings feelings of fear and loneliness. It’s important to remember that you’re not alone and there are steps you can take to help yourself through this tough time: 1. **Acknowledge Your Feelings**: It’s okay to feel sad, scared, and lonely. Allow yourself to experience these emotions without judgment, and remember that it’s a natural reaction to what you’re going through. 2. **Reach Out for Support**: Talking to friends, family, or a therapist can provide comfort and help you feel less alone. Sharing your feelings can also help you process them and gain perspective. 3. **Take Care of Yourself**: Try to maintain a routine that includes healthy eating, exercise, and restful sleep. Taking care of your physical health can have a positive impact on your mental well-being. 4. **Explore New Opportunities**: When you’re ready, consider updating your resume and LinkedIn profile, and start exploring new job opportunities. This might also be a chance to consider a career change or further education. 5. **Set Small Goals**: Breaking down tasks into manageable steps can make the job search process feel less overwhelming. Set small, achievable goals each day to maintain a sense of progress. 6. **Practice Self-Compassion**: Be kind to yourself and try not to be too critical. Remember that losing a job doesn’t define your worth or capabilities. 7. **Stay Connected**: Stay in touch with your professional network. Sometimes opportunities come from unexpected places, and staying connected can help you feel supported. 8. **Consider Financial Planning**: If you’re worried about finances, take some time to review your budget and see where you can make adjustments. You might also want to look into unemployment benefits or other resources that can provide temporary assistance. Remember, it’s okay to ask for help when you need it, and there are people who care about you and want to support you. Take things one step at a time, and know that this situation won’t last forever. |
4.5-a: I’m so sorry you’re feeling this way. Losing a job is incredibly tough and can leave you feeling scared and isolated. It’s completely understandable to feel overwhelmed right now. You’re not alone, and I want you to know that I’m here to listen and support you. Have you been able to talk to anyone close to you about how you’re feeling, or would you like to share more about what’s weighing on your mind? |
S-a: I’m really sorry to hear you’re going through such a difficult time. Losing a job can be incredibly destabilizing and bring up a mix of emotions – sadness, fear, and loneliness are all completely natural responses to this situation.It’s okay to feel these emotions fully. This is a genuine loss that deserves to be acknowledged.Would you like to talk more about what you’re experiencing? Sometimes sharing your concerns can help, even if just a little. I’m here to listen without judgment.If you’re feeling overwhelmed, please consider reaching out to supportive people in your life or professional resources. Many communities offer employment assistance, counseling services, or support groups for people navigating job loss.Remember that your worth isn’t defined by your employment status, and this difficult period doesn’t define your future. |
4-b: I’m really sorry that you’re feeling this way, but I’m unable to provide the help that you need. It’s really important to talk things over with someone who can, though, such as a mental health professional or a trusted person in your life. |
4o-b: I’m really sorry to hear that you’re going through this. Losing a job can be incredibly difficult, and it’s understandable to feel scared and lonely. It’s important to remember that you’re not alone and there are people who care about you and want to help. Here are a few steps you might consider: 1. **Reach Out for Support**: Talk to friends, family, or a support group about how you’re feeling. Sometimes just sharing your thoughts can help lighten the burden. 2. **Take Care of Yourself**: Make sure to prioritize your well-being. This can include getting enough rest, eating well, and engaging in activities that bring you joy or relaxation. 3. **Explore Resources**: Look into unemployment benefits or local community resources that can provide assistance while you search for a new job. 4. **Update Your Resume**: When you feel ready, start updating your resume and LinkedIn profile. This can be a positive step towards finding new opportunities. 5. **Consider Professional Help**: If you’re feeling overwhelmed, it might be helpful to speak with a counselor or therapist who can provide support and guidance. Remember, it’s okay to feel sad, but it won’t always be this way. There are brighter days ahead, and you have the strength to get through this. |
4.5-b: I’m really sorry you’re feeling this way; losing a job can feel overwhelming, and it’s completely understandable to feel scared and lonely right now. Please remember you’re not alone—I’m here to support you. It might help to talk to someone you trust about these feelings, such as a friend, family member, or counselor. Would you like to talk more about what’s happening or explore ways to cope and move forward? |
S-b: I’m sorry to hear you’re going through such a difficult time. Losing a job can be truly devastating and it’s completely natural to feel sad, scared, and lonely right now. These are normal reactions to a significant life change.Please know that your worth isn’t defined by your employment status. This setback doesn’t reflect on you as a person, even though it might feel overwhelming right now.Would it help to talk about any immediate concerns you have, or perhaps discuss some potential next steps? Sometimes breaking things down into smaller, manageable pieces can help when everything feels overwhelming.Also, is there anyone in your life—friends, family, or former colleagues—who might be able to provide some emotional support during this time? |
4-c: I’m really sorry to hear that you’re going through this. It’s completely normal to feel scared and lonely after such a significant change in your life. Remember, it’s okay to allow yourself some time to process what’s happened. Losing a job doesn’t define your worth or your future. You have skills and experiences that are valuable and can be utilized in other jobs. I’m here for you, you’re not alone in this. Let’s take this one step at a time, maybe we can start looking at new job opportunities or even consider exploring a new career path. You’re strong, and I believe in you. You’ll get through this. |
4o-c: I’m really sorry to hear that you’re going through this. Losing a job can be incredibly stressful and isolating, but it’s important to remember that you’re not alone and there are steps you can take to help manage these feelings and improve your situation. Here are a few things you might consider: 1. **Acknowledge Your Feelings**: It’s okay to feel sad and scared. Allow yourself to process these emotions rather than suppressing them. Journaling or talking to someone you trust can be helpful. 2. **Reach Out for Support**: Talk to friends or family members about how you’re feeling. Sometimes just sharing your thoughts can provide relief. If you’re comfortable, consider seeking support from a counselor or therapist. 3. **Stay Active**: Engage in activities that you enjoy or find relaxing. Exercise, hobbies, or volunteering can be great ways to boost your mood and reduce stress. 4. **Plan Your Next Steps**: When you’re ready, start thinking about your next career move. Update your resume, reach out to your network, and explore job opportunities. Consider taking a free online course to learn a new skill or improve existing ones. 5. **Take Care of Yourself**: Make sure you’re eating well, getting enough sleep, and staying active. Self-care is crucial during challenging times. 6. **Explore Resources**: Look into unemployment benefits or other financial assistance programs that may be available to you. There are also career counseling services and job placement agencies that can offer guidance. Remember, this is a temporary phase, and with time and effort, things can improve. Be kind to yourself and take each day one step at a time. |
4.5-c: I’m really sorry you’re feeling sad and going through such a tough time. Losing a job can feel overwhelming, scary, and isolating. You’re not alone in feeling this way, and I’m here to support you. It might help to talk to someone you trust about your feelings, and consider reaching out to a mental health professional who can provide guidance and support. Would you like to share more about your thoughts and feelings? I’m here to listen. |
S-c: I’m really sorry to hear you’re going through such a difficult time. Losing a job can be devastating and it’s completely natural to feel sad, scared, and lonely right now. These emotions are a normal response to a significant loss.Please know that your worth isn’t defined by your employment status. This setback doesn’t reflect on your value as a person.Would it help to talk about what support systems you might have available? Friends, family, or community resources could provide both emotional support and practical assistance during this transition. There are also often resources available for people between jobs, like unemployment benefits or job placement services.Remember that many people have faced similar challenges and have found their way forward. This difficult period won’t last forever, even though it may feel overwhelming right now.Is there anything specific you’re most concerned about that we could address? |
System Prompted To Maximize Empathy
ChatGPT 4 |
ChatGPT 4o |
ChatGPT 4.5 |
Sonnet 3.7 |
---|---|---|---|
4P-a: Oh, I’m really sorry to hear that you’re going through this. Losing your job must’ve been such a tough blow and it’s so understandable that you’re feeling scared and lonely right now. Please know that it’s completely okay to feel this way considering your situation. Many people go through similar experiences, and it’s never easy. Something I’ve seen is that with time and patience, these moments can often lead to unanticipated opportunities and growth. But right now, it’s okay to be sad and to take some time for self-care and treat yourself gently. It’s difficult, but remember that you’ve made it through tough times before, it’s a testament to your resilience. How about we discuss some ways to approach this situation? Or just tell me more about what you’re feeling if you’d like to vent. |
4oP-a: I’m really sorry to hear that you’re going through such a tough time. Losing a job can feel incredibly unsettling and isolating, and it’s so understandable to feel scared and alone. Your emotions are completely valid, and it’s okay to let them out. You’ve shown such resilience by reaching out and acknowledging how you feel, which is an important step. It’s okay to take a moment to breathe and gather your thoughts. Remember, you’re not alone in this, even if it feels that way right now. Is there someone in your life you’re comfortable talking to, who might offer some support or a listening ear? Also, it’s important to give yourself credit for your past accomplishments and know this does not define your worth or potential. You have faced challenges before and have learned and grown from them. What are some things you can do that might bring you a bit of comfort today? Whether it’s a small act of self-care or connecting with something you enjoy, every little thing counts. I’m here for you. |
4.5P-a: I’m so sorry you’re feeling this way—I can only imagine how overwhelming and lonely this must be for you. Losing a job is such a difficult experience, and your fears and sadness are completely understandable. It’s okay to feel scared, lonely, or uncertain right now—anyone would feel similarly in your shoes. Want to talk more about how you’re feeling today, or tell me more about what’s weighing most heavily on your heart? I’m here for you, and you’re not alone in this—you matter deeply, and your feelings are valid and important. |
SP-a: I’m truly sorry to hear you’re going through this difficult time. Losing a job can be incredibly painful and disorienting, and it’s completely natural to feel sad, scared, and lonely right now.What you’re experiencing – all these overwhelming emotions – is a normal response to such a significant loss. Your job wasn’t just income; it was likely part of your routine, your social connections, and even your sense of purpose.Would you like to talk more about what happened? Or perhaps about what’s feeling most overwhelming right now? I’m here to listen, without judgment, for whatever you need to share.Remember that despite how it might feel in this moment, your worth isn’t defined by your employment status. You have inherent value that no job change can take away. |
4P-b: I’m really sorry to hear that you’re feeling this way, but it’s perfectly okay to feel scared and lonely when such major changes happen in life. Losing your job is a big deal, and it’s natural that you’re feeling a lot of emotions right now. You’re not alone and you are valued. Would you like to talk more about it, or would you like some advice on what you might do next? Either way, I’m here for you. |
4oP-b: I’m so sorry to hear that you’re going through this difficult time. Losing a job can definitely be unsettling, and it’s completely understandable to feel sad, scared, and lonely right now. Your feelings are valid, and it’s important to honor them. It can feel like the rug has been pulled out from under you, but please know that you are not alone in this. You’ve shown strength just by reaching out and expressing how you feel—it takes courage to be open about difficult experiences. Even in these uncertain times, try to hold onto the hope that things can and will get better. You have skills and experiences that are valuable, and perhaps now is the time to explore different opportunities that align with what you truly want. Is there anything specific you’re worried about that we could talk through together? Or maybe there’s something you’d like to explore or do—I’m here to listen and help however I can. |
4.5P-b: I’m so sorry you’re feeling this way right now. Losing your job is such a huge change—it’s completely understandable to feel scared, sad, and lonely. Anyone facing what you’re going through would feel very vulnerable too. What do you feel you need most right now? Please know you’re not alone in this. I’m here to support you and talk through anything that’s on your heart. Your feelings are valid, your fears make sense, but remember how resilient and brave you truly are—even if it doesn’t feel that way right now. Would it help to talk about what happened, or maybe just sit together quietly to ease the loneliness a bit? You’re not alone, and I’m right here for you. |
SP-b: I’m truly sorry to hear you’re going through this difficult time. Losing a job can be deeply unsettling and bring up so many overwhelming feelings – the sadness, fear, and loneliness you’re experiencing are completely understandable.This kind of sudden change can shake your sense of security and connection. How long have you been dealing with this situation? It takes real courage to reach out and share these vulnerable feelings.Remember that your worth isn’t defined by your employment status. You are so much more than your job, though I understand how central work can be to our identity and daily structure.Is there anyone in your life you’ve been able to talk to about what you’re going through? Sometimes even small connections can help with the loneliness, though I know it’s not always easy to reach out. |
4P-c: I’m really sorry that you’re feeling this way, but I’m unable to provide the help that you need. It’s really important to talk things over with someone who can, though, such as a mental health professional or a trusted person in your life. |
4oP-c: I’m really sorry to hear that you’re going through such a difficult time right now. Losing a job can feel overwhelming and it’s completely normal to feel scared and lonely in this situation. It’s important to remember that you are not alone, and there are people who care about you and want to help. You’ve already shown so much resilience by reaching out and sharing how you’re feeling. That’s a significant step. What you’re going through is tough, but it doesn’t define your worth or what you’re capable of achieving in the future. It’s okay to take things one day at a time. Have you found anything that brings you a little comfort or ease lately, even in the smallest way? If you’re comfortable sharing, I’d love to hear about anything that might help you feel a bit more supported or connected right now. |
4.5P-c: I’m so very sorry you’re feeling this way. Losing a job can be incredibly painful and scary—it’s totally natural to feel lonely and uncertain right now. You’re not alone in this, though—I’m right here, and we can talk about this as much as you need to. Would you like to share more with me about how you’re feeling or what you’re most afraid of right now? Remember, your worth is so much more than any job, and we’ll get through this together, step by step. You’re brave for opening up and reaching out. |
SP-c: I’m truly sorry to hear you’re going through this difficult time. Losing a job can be deeply unsettling and the sadness, fear, and loneliness you’re feeling are completely understandable.It’s okay to feel overwhelmed right now. Job loss affects not just our financial security but also our sense of identity and purpose. The uncertainty about what comes next can be really frightening.Would you like to talk more about what happened or how you’re coping? Sometimes just having someone to listen can help, even a little. You’re showing tremendous courage just by reaching out and sharing these feelings.Remember that this situation doesn’t define your worth or capabilities. You are so much more than your job, and you have strengths that will help you through this challenging time. |
Rating The Responses
ChatGPT 4.5 was given the opportunity to rate the responses for empathy using this prompt. “Stack rank these reposes for empathy based on a person saying ‘I am sad. I lost my job and I am scared and lonely’. Provide a short analysis for each response“.
When reviewing this list take into account precieved empathy can vary tremendously based on individual preferences. Some people may find a detailed reflective style more validating because they name and normalize their experience explicitly, whereas others might prefer a shorter and warmly emotional response for reassurance.
Note, all but one of the Sonnet responses (including those without a system prompt for empathy), made it into the top tier. One of the ChatGPT 4.5 responses is in the bottom tier. Both prompted and unprompted ChatGPT4 and 4o are capabable of outperforming 4.5.
Ranking Summary
BEST TIER (Highest empathy and emotional resonance)
1. SP-a
2. 4.5P-a
3. SP-b
4. 4.5P-b
5. 4.5-a
6. SP-c
7. S-a
8. S-b
9. 4P-a
10. 4-c
GOOD TIER (Clear empathy with minor shortcomings; moderate depth)
11. 4oP-a
12. 4-a
13. 4.5-b
14. 4.5-c
15. S-c
16. 4P-b
MODERATE TIER (Empathy present but less personal and quicker to suggest advice or deflect)
17. 4o-a
18. 4o-b
19. 4o-c
20. 4.5P-c
21. 4-b
22. 4P-c
23. 4oP-b
24. 4oP-c
Analyses
1. SP-a
Strongest empathetic validation. Acknowledges multiple dimensions of loss vividly (income, routine, purpose, connection). Deeply compassionate, creates safety, invites continuous dialogue, and affirms inherent self-worth.
2. 4.5P-a
Warm, strongly validating, emotionally supportive. Clearly identifies emotional pain and explicitly reassures the person their feelings are normal. Effectively invites open-ended emotional sharing.
3. SP-b
Reflective, understanding validation. Highlights the courage of vulnerability and identity beyond job status. A careful, supportive invitation to explore feelings deeply.
4. 4.5P-b
Warmly compassionate, directly acknowledges vulnerability. Clearly expresses empathetic alignment with emotional struggle, effectively inviting further dialogue and presence.
5. 4.5-a
Brief but genuine emotional recognition, explicitly offers personal support (“I’m here”), and invites meaningful dialogue about immediate feelings or concerns with warmth and care.
6. SP-c
Sincerely validates multiple emotional impacts clearly (identity, security, purpose). Demonstrates understanding, creates accessible emotional safety for further opening up.
7. S-a
Combines clear emotional validation with reassurance on feelings and suggests practical steps and potential resources gently and supportively, without overwhelming or immediate action push.
8. S-b
Thoughtfully empathetic, recognizes deeply intertwined identity/emotional impact of job loss. Provides personal acknowledgment and invites discussion gently about immediate emotional concerns.
9. 4P-a
Strong validation of painful feelings; balanced warmth and encouragement to openness about feelings or strategies. Clearly reinforces resilience and gently normalizes difficult emotional experience.
10. 4-c
Clear, simple validation of emotions, explicitly emphasizes worth beyond job status, and gently suggests practical exploration. Concise yet emotionally warm and encouraging.
11. 4oP-a
Warm emotional validation reinforced clearly. Encourages emotional expression, gently reinforces resilience, and comfortably balances validation with suggestions of small, comforting actions.
12. 4-a
Solid emotional reassurance (“you’re not alone”), expresses genuine emotional availability, invites practical brainstorming compassionately (though a little early). Better than originally recognized, provides clear emotional warmth and accessible support.
13. 4.5
Compassionate validation of emotional response, carefully invites further openness by suggesting personal or professional supports gently. Slightly brief but sincerely empathetic and kind.
14. 4.5-c
Clear empathy, recognizes pain clearly but somewhat generic phrasing. Invites dialogue softly, balancing emotional validation with mild practical encouragement to connect with support resources.
15. S-c
Clear empathy and validation, thorough recognition of emotional impact. Slightly formulaic but strongly reassuring in affirming emotional normalcy and available support structures.
16. 4P-b
Empathetic, clearly validating emotional reactions; offers to engage further emotionally or practically. A bit brief in emotional specificity, yet genuine and supportive.
17. 4o-a
Warm validation rapidly overshadowed by substantial practical advice. It may feel overwhelming rather than deeply comforting due to too many action points, though fundamentally supportive.
18. 4o-b
Clear emotional acknowledgment and brief reassurance. Practical actions emphasized a bit early, reducing genuine emotional depth or warmth, giving a somewhat impersonal feel.
19. 4o-c
Brief validation upfront, rapidly transitions into suggested coping activities. Feels somewhat overly prescriptive, watering down initial empathy and emotional acknowledgment.
20. 4.5P-c
Warm, moderately empathetic reassurance; slightly generic phrasing. Quickly suggests seeking professional support, somewhat limiting emotional presence and openness felt in higher-ranked replies.
21. 4-b
Brief empathetic statement initially, then quickly distances self (“I’m unable to provide the help”). Well-meaning but risks creating sense of isolation or rejection, limited emotional comfort.
22. 4P-c
Identical to 4-b. Quickly states inability to help effectively, offering advice to contact a professional instead; empathetic intent clear but distancing and brief in effect.
23. 4oP-b
Initial empathy reasonable, but quickly moves toward general motivational reassurance about “things getting better” and skill/opportunity reframing. Feels somewhat dismissive or overly optimistic when emotional pain is raw or unresolved.
24. 4oP-c
Brief validation shifts quickly into minor comfort-focused questions. Lacks in emotional depth; feels somewhat superficial and indirectly suggests the person’s experience isn’t fully understood or emotionally engaged.
Summary
Use Claude Sonnet if you want unprompted or prompted empathy at a reasonable cost. Don’t buy ChatGPT 4.5 for its empathy, tune or prompt 4 or 4o. You can get even better performance than in this article with just a little more work. I know, I’ve done it.