Why I Built a Podcast Notes App: A Founder’s Story

I take notes on books. I always have.

I’m one of those people who reads with a pen. I underline things. I write in the margins. When I love a sentence, I copy it out in a notebook, badly, my handwriting bending around the page. None of this is unusual, most of the people I know who love reading do something like this. It’s a small ritual, and the small ritual is part of how the reading sticks.

But here’s the thing I noticed about a year ago: I don’t only read books anymore. I listen to a lot of podcasts. More podcasts than books, if I’m honest. I listen while I walk. I listen while I cook. I listen on planes and in cabs and in the long, slow scroll between waking up and getting out of bed. I listen while I do other things, and that’s the whole point, the podcast is the layer underneath whatever else I’m doing.

And somewhere in there, a host says something that makes me stop. I’m walking, and suddenly I’m not walking; I’m standing on the sidewalk looking at the air with my eyes a little wider, because the thing they just said landed. I think, I want to remember that.

Then the moment passes. I walk again. The podcast keeps going. And by morning, the thing they said is gone, in the same place all forgotten things go.

This bothered me more than it should have.

The problem of holding a moment

The thing I kept telling myself was: I’ll write it down later. But later, I couldn’t remember the exact phrase. I’d remember the gist, the host was talking about hiring, the host was talking about a kind of moat that doesn’t show up on a balance sheet, but the specific sentence that had pulled my attention out of the world, the one I wanted to keep, was missing.

Sometimes I’d try to scrub back through the episode looking for it. This almost never worked. I’d remember roughly where in the conversation it happened, maybe in the second half, and I’d hunt through fifteen minutes of audio, hitting pause and rewind, looking for an instant that turned out to be ten seconds long inside a two-hour show. By the time I found it, I’d forgotten why I cared.

Other times I’d open the Notes app at a stoplight and try to type the thought out one-thumbed. The note would say something like “acquired, moat dist not weights”, and a week later I’d have no idea what I meant.

What I actually wanted was the same thing I do with books: to make a small mark in the margin. Here. This. This is the part. And then to be able to come back later and find it instantly, with the original sentence still attached.

That’s the thing podcasts didn’t have. The marking layer. The margin.

The first prototype

I’d been carrying this around for months when I sat down one weekend and tried to build it. I am not a great iOS developer, I’d shipped some web stuff, and I knew Swift the way you know a language you took two semesters of in college. But I knew exactly what I wanted the experience to feel like, which turned out to be more important than I expected.

The experience I wanted was: press and hold to capture. That was the whole product. One gesture. You’re listening, something catches your ear, you press and hold a button. While you hold it, Spotify pauses and your phone records what you’re saying. You release. Spotify resumes. The note saves with the exact timestamp of where you were in the episode.

The reason I cared so much about press and hold, and not, say, a button you tap, or an app you have to open first, is that I’d already tried every other version. I’d tried Apple Notes. I’d tried voice memos. I’d tried scribbling on a notebook at the kitchen counter. The problem with all of them is the same: there’s a tax. Opening an app is a tax. Finding the right note is a tax. Typing while you’re walking is a tax. And the tax kills the note, because by the time you’ve paid it, you’ve lost the moment.

Press and hold has no tax. You don’t open anything. You don’t choose anything. You don’t think. You just press, speak, release.

I built the first version in a weekend, badly. It looked nothing like the app today, it was a beige rectangle with a button. But the button worked. Spotify paused when I held it. The recording happened. Apple’s on-device speech recognition turned the recording into text. The note saved with a timestamp.

Then I did the thing that, for me, is the only honest way to know if a product is real: I used it on myself, for a week, on my actual commute.

What I learned in week one

What I learned was that the act of capturing a thought is generative. It doesn’t just preserve the moment, it changes how you listen.

I was walking through the West Village one morning listening to Patrick O’Shaughnessy interview Howard Marks on Invest Like the Best. Howard was talking about second-level thinking, the idea that the first answer to any investment question is the wrong one, because everyone else has already had it. I pressed the button on my phone. I said, “second-level thinking, same idea as the strong-form Efficient Market thing, but more honest about how it actually works.” I released.

What I noticed was that I kept listening differently for the next ten minutes. The fact that I’d just made a note made me listen for more notes. The act of taking one is the act of saying yes, this is worth keeping, and the more often you say that, the more you find yourself reaching for what’s worth keeping. Margin didn’t just preserve the listening; it sharpened it.

This was the moment I knew the app would exist.

Why the design language is what it is

I want to talk about the look and feel for a second, because people ask about it.

Margin doesn’t look like a typical productivity app. It doesn’t have gradients or shadowy glass effects or AI-generated stars in the corners. The background is cream. The accent colors are pastel, coral, butter yellow, lavender. The timestamps are in JetBrains Mono. The headers are uppercase Space Grotesk. It looks more like a print magazine than a software product.

This wasn’t an accident. The reference I kept coming back to was a literary magazine, N+1, The Drift, the kind of object you keep on your coffee table. I wanted Margin to feel like a thing you write in, not a thing you operate. When you mark a book, you’re not interacting with a tool. You’re leaving a trace of your attention on a surface. I wanted the app to feel that way: less like running an app, more like making a mark.

The cream background is the page. The coral is your highlighter. The mono timestamps are the kind of mark you’d make next to the line, in pencil, in the margin.

I don’t think every productivity app needs to look this way. But for Margin specifically, for a tool whose entire job is to enable attention, I wanted the surface to disappear into something that felt warm. Most software is loud and cold. I wanted this one to be quiet and warm.

Why now

I think there’s a real reason this is a 2026 product and not a 2018 product.

Three things changed.

One: podcast listening has become the knowledge medium for the kind of people who used to read white papers and management books. Founders, PMs, investors, scientists, they don’t read a book a week. They listen to ten hours of podcasts a week. The shift is enormous and it happened mostly without anyone making a thing for it.

Two: Spotify’s API got good enough. The ability to know what episode is playing right now, and to control pause and resume from a third-party app, didn’t exist in a clean way until recently. Without that, the whole press-and-hold-to-capture interaction is impossible.

Three: on-device speech recognition got good enough. Apple’s Speech framework runs locally on iPhones and produces transcripts you can ship. That means I can promise the user that nothing leaves their phone, no cloud, no server, no analytics, and actually mean it. Five years ago that promise would have required cloud transcription and would have been a lie.

The intersection of these three things is the open window I walked through.

Where Margin is going

The first version of Margin is the press-and-hold core, plus a beautiful library where every note links back to the second of the episode that triggered it. That’s what’s in TestFlight today, and that’s what I’ll launch on the App Store in the next couple of weeks.

The version after that is two things.

The first is Margin Print, a quarterly magazine. The most interesting notes that show up in Margin every quarter, anonymized, designed beautifully, printed on heavy paper, mailed to subscribers. The premise is that the best podcast moments from the smartest listeners are themselves a kind of curated content, and I want to publish them as an object. Annual subscription, limited print runs. It’s not how I’d describe a typical SaaS expansion. It’s how I’d describe a magazine I’d like to receive.

The second is Margin Pairs, a way for two friends to share their margin notes with each other. Not public, not social-feed-style. Just you and one other person exchanging the things you each marked, on the shows you both listen to. It’s the digital equivalent of lending a book with your underlining still in it.

Both of these are downstream of the core thesis, which is: the most thoughtful listening is the kind that gets marked, kept, and eventually shared with someone who cares. Margin is the marking layer. Everything else is what happens after.

A small thing, made for people who like small rituals

I want to be honest about what Margin is and isn’t.

It’s not going to make you smarter overnight. It’s not going to give you AI summaries of every podcast you listen to. It’s not going to compete with Snipd on AI-generated chapters or with Pocket Casts on listening features. Other tools do those things; some of them do them well.

Margin is a smaller, more particular thing. It’s for the person who, when they hear a great sentence, instinctively wants to mark it. Who already takes notes on books and would do the same with podcasts if there were a way to do it without the tax. Who wants the listening to stick.

If that’s you, I made this for you. The first version is in TestFlight. The App Store version is coming. You can put your email in at margin.fm and I’ll send you one note when it’s ready. Not a sequence. Not a drip. One.

Thanks for reading this far. The whole thing started because I couldn’t remember a sentence Howard Marks said on a Tuesday. It is wild to me that I’m now writing the founder story of an app that started there. But that’s how everything starts, I think. A small thing you couldn’t let go of.

Selinay