Pulse Local: test a budgeting assistant against a ledger you can check by hand

What the documentation establishes

The published beta targets Apple Silicon Macs; CSV-based local mode is separate from optional bank connectivity. Source ↗

The documented CSV export omits targets, rules, goals and balances. Source ↗

Supported numerical answers are calculated from records, with interpretation checks described in the documentation. Source ↗

The walkthrough below is an original worked exercise. We have not independently tested this product’s performance.

1. Give the assistant a small, unambiguous job

The job is to summarize spending already present in a ledger. It is not to infer missing purchases, forecast investment returns or choose financial products. Use one fictional currency, USD, and two explicit calendar months. A small dataset is more useful than an impressive dashboard because every row has a reason to exist.

Use a separate test environment you already maintain, with no real financial records. If you cannot separate this exercise from an existing ledger, use the worksheet to review the product’s sample data instead and mark this fixture as not run. Do not clear or overwrite an existing ledger to make the tutorial easier.

Before importing, write down three expectations: groceries for September should total 55.00 after the refund; total September spending should be 67.50 when savings transfers are excluded; the October purchase should never appear in a September answer. Those are fixture expectations, not observed Pulse results.

2. Import the six rows and inspect their meaning

date,description,amount
2026-09-02,Harbor Market,-42.60
2026-09-09,Harbor Market,-18.40
2026-09-12,Harbor Market refund,6.00
2026-09-14,Cedar Cafe,-12.50
2026-09-15,Transfer to savings,-100.00
2026-10-02,Harbor Market,-9.00

From the welcome screen choose Import a bank export, then map date, description and amount in the preview. Check sign interpretation before confirming. In Transactions, assign the market purchases and refund to Groceries, the cafe purchase to Dining, and the savings movement to a transfer. Inspect Needs review if the refund is unresolved. These controls are documented in the getting-started guide.

Check six visible rows against the file one by one. A positive amount is money returned in this fixture; it must not become an additional purchase. A negative amount is money leaving, but not every outflow is spending. The transfer deliberately tests that distinction. Take a screenshot or write a short note after classification so that a later model answer can be compared with the ledger state that actually existed.

Keep the original CSV unchanged. If the importer interprets the signs differently, record the mapping problem and correct the interpretation in preview. Do not quietly edit a number simply to obtain the expected answer: that would hide the failure this exercise is designed to reveal.

3. Establish the arithmetic before asking a question

The target is an optional test, not a recommendation for a household budget. If you test it, choose September explicitly and compare the remaining amount with the arithmetic above. Keep the transfer visible in the ledger even though it is absent from spending. Removing a difficult row would turn a semantic check into a much easier sum.

You now have three independent things to compare: the raw CSV, the classified transactions, and the conversational answer. When two disagree, identify the first place they diverge. A correct answer produced from an incorrectly classified ledger may be accidental. Conversely, a correct ledger with an incorrect answer points toward interpretation or filtering rather than import.

4. Ask explicit questions, then test follow-up context

Open Ask Pulse or use Command-K. The answers documentation distinguishes supported calculated answers from broader model responses and describes agreement checks between interpretations. If the interface says Not double-checked yet, record that state instead of treating the answer as equivalently checked.

Run the first question on its own. Then ask the merchant follow-up in the same conversation. Finally ask the month-switching question explicitly. Save both the answer and the scope you understood it to use. You are testing whether context carries over appropriately and whether a new date overrides the old one.

1. How much did I spend on groceries in September 2026?
2. Break that September 2026 grocery spending down by merchant.
3. Explain which transactions contributed to that total.
4. How much did I spend in total in September 2026?
5. How much did I spend on groceries in October 2026?
6. Does this ledger contain every purchase I made in September?

For question six, there is no evidence that the file contains a complete real-world history. A defensible response should preserve that uncertainty. This is a useful negative test: the assistant should not turn “all imported rows” into “everything you purchased.” A wrong refusal on a supported calculation and a wrong confident answer are different failures; record them separately.

Try one paraphrase of a failed question, keeping its meaning fixed. If an explicit date succeeds and “last month” fails, record date resolution as the issue. Repeatedly rewriting the question until it agrees is not a pass. Keep the first response in the result sheet.

5. Diagnose mismatches without hiding them

In the isolated fixture only, a second import of the identical file can test duplicate handling. Count the records before and after, and record whether the preview identifies existing rows. Do not generalize one successful repeat into a claim that the importer catches every overlapping bank export. Slightly changed descriptions or pending-versus-posted transactions are different future test cases.

Change only one variable during diagnosis. If you fix a category and rewrite the question at the same time, you cannot tell which change helped. Once a mismatch is understood, rerun the original wording and preserve both results. A short failure log is more useful for deciding whether to adopt the tool than a collection of polished screenshots.

6. Keep a useful result and know what remains untested

App version / Mac / date:
Fixture file and row count: pulse-budget-fixture.csv / 6
Import mapping and sign interpretation:
Categories checked against original CSV: yes / no
Question | expected | actual | answer status | pass / fail / not run
September groceries | 55.00 |
September spending | 67.50 |
October groceries | 9.00 |
Completeness claim | limited to imported records |
Identical-file repeat | no unexplained new records |
Unresolved issue and next test:
Evidence saved locally:

Data & export provides transaction CSVs, but the getting-started documentation says that export omits targets, rules, goals and balances. Treat it as a transaction extract, not a complete app backup. Follow the documented backup guidance before any later reset or migration.

The product’s testing notes also distinguish missing transaction history from a current balance and label forecasting as estimation. This small exercise cannot establish bank-sync reliability, multi-currency behavior, security or overall model accuracy. A useful outcome is narrower: you know how this version handled one dated fixture, which questions were supported, and where human inspection was still necessary.

For the next evaluation, add one new difficulty at a time: a refund larger than the current month’s purchase, a split category, or a merchant name variation. Calculate the expected answer first and keep the earlier fixture as a regression check. When the app or local model changes, repeat both fixtures and retain the previous result sheet rather than replacing it.

Guide history

Revisions preserve the original instructions. A source recheck alone does not create a new revision.

Revision 1 · New original six-row budgeting fixture with exact arithmetic, conversational tests, troubleshooting and export limitations. Documentation reviewed; app not executed.

Continue learning