Notetaker 2.0: Letting AI Triage My Zoom Calls and Email

I had the same problem in two places at once.

The first was Zoom. Claude Code is connected to my Zoom account, so it can pull down any recording and read the transcript. But my calls are all over the map. One might be a tutorial I am recording. The next is an internal working session with my team. Then an onboarding call, a sales call, or a meeting I started by accident that is six seconds long. Every one of them is titled "Personal Meeting Room," so the title tells you nothing.

The second was my inbox. Some emails are from real people who are waiting on me. Most are notifications, receipts and promotions. A few need me to make a decision or pay something.

Both are piles where every item could be anything. When I explained what I was building to a client on a call, he called it "Notetaker 2.0." That is a good name for it.

💡 Why This Matters

AI notetakers pull action items from every call as if every call were a meeting. Inbox tools sort by sender. Neither one asks the first question: what kind of thing is this, and what should happen next for this kind of thing? Answer that first and the rest gets much easier.

The Old Way

Before this, I used Claude on a one-off basis. "We just had an onboarding call, go grab the transcript and do the onboarding stuff." That works, but only when I remember to ask, and only for the calls I think about. Most calls ended and nothing happened with them.

Email was the same. I would scroll, skim, and hope I caught the ones that mattered.

The Pattern

Both systems use the same four steps:

An event fires. A Zoom transcript finishes, or new mail arrives.
Claude classifies it. One call to Claude with a rules file that describes the categories.
The category picks a playbook. An onboarding call needs different follow-up than a tutorial.
Claude stages the next step and does not act. It drafts, suggests and emails me. I approve.

The fetching, scheduling and emailing are plain scripts. Claude only does the part that needs judgment, which is reading the thing and deciding what it is. That split is the same idea from why I split my AI workflow into two layers.

The Zoom Call Classifier

When Zoom finishes transcribing a cloud recording, it sends an event. My server listens for that event and hands the transcript to Claude. An hourly sweep catches anything the event missed, and transcripts under 300 words get skipped, so the six-second accidental meetings never cost anything.

Claude sorts each call into one of these categories:

Category	What it is
Tutorial	Me recording a walkthrough or demo for an audience
Client meeting	A working session or check-in with an existing client
Client onboarding	The first sessions after a client signs
Sales discovery	A prospect or early conversation
Internal working session	Day-to-day planning, building and deciding
Team meeting	Three or more internal people
Networking	A peer, partner or vendor

Each category has a playbook. A tutorial should become a title, chapters, clips and maybe a blog post. A client onboarding should fill in that client's knowledge base, list the access we still need, and put the first milestones on the calendar. A sales call should update the CRM and draft a follow-up that moves them to the next step.

Then two things happen:

The recording gets renamed. "Personal Meeting Room" becomes something like "[Client Meeting] Client Name: Ad account access, website scope, next steps." I can finally scan my recordings list and know what each call was.
I get an email. It says what kind of call it was, who was on it, a short summary, and the five things it would have done next, each with the line from the transcript that backs it up.

That last part is on purpose. It is in practice mode. It tells me what it would do, and I check whether it is right before it is allowed to do any of it.

Want something like this built for your business? Create a ticket today →

Our Toolbox. Every platform, plugin, and service we use and recommend, in one place. See the list →

The Email Classifier

The email version runs every four hours across both of my inboxes, and each inbox gets its own rules file. Claude sorts every new email into one of three buckets:

Needs me. A real person is waiting, money or an account is at risk, a deadline is coming, or it is a decision only I can make.
FYI. Useful to know, nothing to do. Receipts, payments that already went through, confirmations.
Noise. Promotions, newsletters, cold pitches, social notifications.

The test I gave it for the gray area is one question: if I never saw this email, would something bad or awkward happen? If yes, it needs me.

It runs in two stages. The first pass is a quick sort of everything and drafts nothing, so it stays cheap. Only the emails that need me get a second, deeper look. That pass can read the whole thread, check who the person is, and pull context from my CRM. Then it writes a reply and saves it in Gmail as a draft. It never sends.

Everything lands in one digest email. Each item has a button to open the draft and a button to add it to my to-do list. If nothing needs me, no email gets sent at all.

Teaching It With Examples

The part I like most is how it gets smarter. I did not know all the categories when I started, and I still do not. So the rules files grow from real examples.

When it calls something "needs me" that I do not care about, I tell it, and that example goes into the rules. A payment I already sent is FYI. An order confirmation is FYI. A Google Search Console indexing problem needs me, with the property and the fix spelled out, because that one goes on my to-do list.

It works like training a new assistant. You do not hand them a perfect manual on day one. You correct them on real examples until they know how you think.

Downsides

It is still in practice mode. The Zoom side renames recordings and emails me, but it does not file anything into the CRM or task tracker yet. That is on purpose until I trust it.
It will get some wrong. That is the reason for practice mode and for drafts instead of sends.
Your calls and emails go to the AI. Be thoughtful about what that means for your clients and your business before you turn something like this on.
Time. I built both of these in one day. The Zoom side went from first script to auto-renaming in a little over two hours. The email side took the rest of the afternoon and evening, and the rules files keep growing.

Sort First, Then Grow

Zoom calls and email are hard to automate for the same reason. The inputs are all over the place. You cannot write one rule that says "do this with every call" or "do this with every email," because no single next step fits all of them.

What got me moving was a smaller question: what should happen with every Zoom call, and every email? The first answer did not need to be a full list of actions. It was "at least look at it and figure out what kind of thing it is." That much I could build.

So I built the sorter first and let it practice. It reads, categorizes, renames and drafts, and it tells me what it would have done. I love opening Gmail and finding the replies already sitting in my drafts.

Now that it is working, the next steps are easy to see. For each category I can add the follow-up I want. Client onboarding fills in the client's knowledge base. Tutorials turn into clips and posts. One category at a time.

The big win is that I made it real. These are no longer ideas I talk about on calls. They run on their own, and they are more autonomous than anything I had before. Every new playbook moves them a step further.

If your calls or your inbox feel like a pile where everything could be anything, start with the sorter. Write down the five or six kinds of things that show up, let Claude practice sorting them, and add the next steps one category at a time.