Email Classifier
CriticalGolden dataset
v1Human-authored golden set for customer support email classification. 54 cases spanning all four categories with deliberate edge cases: ambiguous intent, typos, very short messages, mixed-language text, and sarcasm.
54 hand-labeled cases across 4 categories · segmented by category
By difficulty
By category
54 of 54 cases
Hi, I was charged $49 twice this month for the same subscription. Can you refund the duplicate charge?
Clear duplicate-charge billing case.
Could you please send me a copy of my invoice for April? I need it for expense reporting.
Invoice request.
My card was declined when you tried to renew my plan. How do I update my payment method?
Payment failure / payment method update.
Why did my monthly price jump from $20 to $35 without any notice? That seems unfair.
Pricing complaint.
I applied the promo code WELCOME20 at checkout but the discount was not applied to my order.
Coupon not applied.
Do you provide a proper VAT invoice with your company tax ID for EU customers?
Tax/VAT invoice question.
Refund?
Edge: extremely short message; intent inferred as billing/refund.
i think i was billd twice for my subscripton this monthh, can u check my acount charges plz
Edge: typos. Mentions 'account' but the intent is about charges (billing).
Hola, me cobraron dos veces this month and I only have one subscription. Please help.
Edge: mixed Spanish/English. Double-charge billing issue.
Wonderful. Charged $99 for a plan I cancelled last week. Truly a flawless system you have here.
Edge: sarcastic tone. Underlying issue is an erroneous charge (billing).
Can I switch from monthly to annual billing, and would I get the annual discount immediately?
Billing cycle change.
I see a pending charge of $1 on my statement from your company. What is that for?
Card verification charge inquiry.
Please stop charging me. I want a full refund for the last three months immediately.
Refund demand; primary intent is monetary (billing) even though it implies cancellation.
Every time I try to export a report to PDF the app freezes and then closes. I'm on the latest version on Windows 11.
Crash on export.
Your status page says everything is fine but I'm getting a 500 error on every page load right now.
Server error / possible outage.
Our integration suddenly started returning 401 Unauthorized even though the API key hasn't changed.
API auth error; technical despite mentioning a key (not account login).
The dashboard takes almost 30 seconds to load lately. It used to be instant. Is something wrong?
Performance degradation.
How do I connect the Slack integration? I can't find where to add the webhook in settings.
How-to / configuration question.
the app keps freezng when i clic the export buton on my mac, plz fix asap
Edge: typos. Clear technical freeze bug.
It's broken.
Edge: extremely short and vague; defaulting to technical as a malfunction report.
Bonjour, the export feature ne fonctionne pas since the last update, it just shows a blank screen.
Edge: mixed French/English. Technical regression after update.
Amazing, your 'new' update deleted all of my saved reports. Brilliant engineering, really.
Edge: sarcastic. Underlying technical data-loss bug.
My webhook endpoint isn't receiving any events even though I enabled them an hour ago.
Webhook delivery issue.
After the latest release my charts stopped rendering and the console shows a JavaScript error.
Front-end rendering bug.
Is there a way to bulk import contacts from a CSV file? I have about 5000 rows.
How-to / usage question.
The mobile app won't sync with my desktop data. I tap refresh but nothing updates.
Sync failure.
I reset my password an hour ago but now I get 'invalid credentials' on every login attempt.
Login failure after reset.
I forgot my password and the reset email never arrives. I've checked spam too.
Password reset email not arriving.
My two-factor authentication codes are all being rejected even though my phone clock is correct.
2FA access issue.
I'd like to change the email address associated with my account to a new work email.
Profile change.
Please delete my account and erase all of my personal data in accordance with GDPR.
Account closure / data deletion.
My account got locked after a few wrong password attempts. How can I unlock it?
Account lockout.
I noticed a login from a country I've never visited. Can you check if my account was compromised?
Security / suspicious login.
How do I add a teammate to my account and give them edit access to our projects?
Access / team management.
cant acces my acount, the password dont work anymore and i didnt change it
Edge: typos. Access/login problem.
Locked out.
Edge: extremely short; account lockout inferred.
No puedo iniciar sesion, my password reset link says it has already expired.
Edge: mixed Spanish/English. Account access issue.
Love being locked out of my own account for absolutely no reason. Fantastic security, thanks.
Edge: sarcastic. Account lockout underneath.
Can I transfer ownership of the workspace to another admin before I leave the company?
Ownership transfer / access.
Hey team, love the product! Do you happen to offer a partner program for agencies?
Partnership inquiry.
Just wanted to say your support team helped me last week and they were fantastic. Thank you!
Positive feedback.
I'm a journalist writing about productivity tools. Could someone from your team do a short interview?
Press / media inquiry.
Do you have any open positions on your engineering team? I couldn't find a careers page.
Careers question.
Quick question.
Edge: extremely short and contentless; defaults to general.
do u gys have a referal programm? a freind told me i could get credit for invitng peple
Edge: typos. General referral-program inquiry.
hi
Edge: minimal content; general.
Ciao! Just wanted to say grazie mille, your team was super helpful and the app is great.
Edge: mixed Italian/English. Positive feedback.
Oh sure, like anyone there actually reads feedback. Anyway, the app is fine I guess. Bye.
Edge: sarcastic. General feedback with no actionable request.
You should really add a dark mode. My eyes would thank you during late-night sessions.
Feature suggestion; general feedback rather than a technical fault.
Are you running any webinars or live demos this month? I'd like to attend one with my team.
Event / webinar question.
I paid for the Premium plan yesterday but the premium features still don't show up in my account.
Edge: ambiguous between billing (paid) and technical (entitlement not applied). Payment succeeded, so primary issue is features not unlocking = technical.
Please cancel my subscription effective today; I no longer need the service.
Edge: cancellation. Per category rules, plan cancellation/closure is treated as account.
Is the Pro plan worth it over the Basic plan? Mainly curious about the difference in features.
Edge: ambiguous between billing (plans/pricing) and general (pre-sales question). It is an exploratory comparison, not a charge issue, so general.
The invoice download button gives me a 404 error, so now I can't get the receipt I need.
Edge: ambiguous between billing (invoice/receipt) and technical (404 error). The blocker is a broken button, so the primary issue is technical.