CLI-015: Keep tabs and separator characters from reshaping the terminal cards

docs/backlog/INDEX.md

@@ -221,7 +221,6 @@ obsolete), then `priority` (P0 → P3).
 | [CAP-023](./capture/CAP-023-fill-the-feed-reader-gaps-discovery-spellings-user-agent-redirects-timeouts-and-429.md) | Fill the feed reader gaps (discovery, Medium spellings, user agent, redirects, timeout, 429, empty feed) | P2 | M | capture | ready | CAP-005, CAP-008, CAP-018, CAP-020, CAP-026, XC-054 | — | — | — |
 | [CAP-025](./capture/CAP-025-a-timed-out-sign-in-and-a-vendor-refusal-read-the-same-and-the-vendors-reason-is-lost.md) | Bug: a timed-out sign-in and a refusal read the same, and the vendor's reason is thrown away | P2 | S | capture | ready | CAP-009, CAP-024, XC-057, APP-178, XC-025 | — | — | — |
 | [XC-066](./cross-cutting/XC-066-check-a-feedback-requests-size-and-shape-before-it-spends-shared-limits.md) | Check a feedback request's size and shape before it spends the relay's shared limits | P2 | S | cross-cutting | ready | XC-028, XC-031, APP-058, XC-065 | — | — | — |
-| [CLI-015](./cli-ux/CLI-015-keep-tabs-and-separator-characters-from-reshaping-the-terminal-cards.md) | Keep tabs and separator characters from reshaping the terminal cards | P2 | S | cli-ux | ready | CLI-013, XC-019 | — | — | — |
 | [MON-032](./monetization/MON-032-prepare-for-stripe-holds-on-seller-accounts.md) | Prepare sellers and support for Stripe payout holds and account reviews | P2 | S | monetization | ready | MON-030, MON-031 | — | — | — |
 | [MON-011](./monetization/MON-011-verify-deployed-code-is-actually-live-after-deploy.md) | Verify the Worker is actually running newly deployed code, not a warm stale instance | P2 | S | monetization | ready | MON-006 | — | — | — |
 | [APP-224](./desktop-app/APP-224-reconnect-an-existing-store.md) | Reconnect a store this Mac didn't set up | P2 | M | desktop-app | ready | APP-223, XC-069, APP-215 | — | — | — |
@@ -411,6 +410,7 @@ obsolete), then `priority` (P0 → P3).
 | [APP-203](./desktop-app/APP-203-a-row-menu-is-never-cut-off.md) | A row menu is never cut off | P1 | XS | desktop-app | completed | APP-200, APP-201 | — | — | — |
 | [APP-204](./desktop-app/APP-204-a-row-menu-closes-on-a-click-elsewhere.md) | A row menu closes on a click elsewhere | P1 | XS | desktop-app | completed | APP-200, APP-203 | — | — | — |
 | [XC-128](./cross-cutting/XC-128-fix-the-two-flakes-main-hit-after-the-test-pyramid.md) | Fix the two flakes main hit after the test pyramid | P1 | S | cross-cutting | completed | XC-106, APP-270, XC-124, XC-109 | — | — | — |
+| [CLI-015](./cli-ux/CLI-015-keep-tabs-and-separator-characters-from-reshaping-the-terminal-cards.md) | Keep tabs and separator characters from reshaping the terminal cards | P2 | S | cli-ux | completed | CLI-013, XC-019 | — | — | — |
 | [MON-057](./monetization/MON-057-say-no-way-to-be-paid-on-a-first-deploy-not-worker-not-found.md) | Say "no way to be paid" on a first deploy, not "Worker not found" | P2 | S | monetization | completed | MON-047, XC-069, XC-085, MON-024 | — | — | — |
 | [CAP-031](./capture/CAP-031-end-a-hosted-sign-in-only-on-an-answer-not-on-any-request-to-the-callback.md) | End a hosted sign-in only on an answer, not on any request to the callback address | P2 | XS | capture | completed | CAP-009, CAP-030, APP-120 | — | — | — |
 | [CAP-038](./capture/CAP-038-collect-every-save-waiting-for-an-agent-not-only-the-oldest-200.md) | Collect every save waiting for an agent, not only the oldest 200 | P2 | S | capture | completed | CAP-036, CAP-039, XC-067, APP-207 | — | — | — |

docs/backlog/cli-ux/CLI-015-keep-tabs-and-separator-characters-from-reshaping-the-terminal-cards.md

@@ -4,13 +4,13 @@ title: Keep tabs and separator characters from reshaping the terminal cards
 priority: P2
 effort: S
 component: cli-ux
-status: ready
+status: completed
 related: [CLI-013, XC-019]
 blockers: []
 dependencies: []
 github_issue: null
 created: 2026-10-14
-updated: 2026-11-11
+updated: 2026-11-13
 ---
 
 ## Problem
@@ -49,18 +49,18 @@ and the same tab rule for diff lines (CLI-013 adds the cleaning there).
 
 ## Acceptance criteria
 
-- [ ] A tab in each of the title, metadata, teaser, useful if, not useful
+- [x] A tab in each of the title, metadata, teaser, useful if, not useful
       if, free sample and body prints as one or more spaces on both cards,
       never as a raw tab and never as nothing.
-- [ ] `\x0b`, `\x0c`, `\x1c`, `\x1d`, `\x1e`, `\x85`, `\u2028` and `\u2029` in a
+- [x] `\x0b`, `\x0c`, `\x1c`, `\x1d`, `\x1e`, `\x85`, `\u2028` and `\u2029` in a
       body do not start a new line on either card, and none of them is
       printed.
-- [ ] `a\rb` in a title prints `ab`. In a body it prints as two lines, which
+- [x] `a\rb` in a title prints `ab`. In a body it prints as two lines, which
       is today's behaviour, kept so that a lone carriage return can never
       move the cursor back over a line.
-- [ ] A memory changed only by one of the characters above shows the same
+- [x] A memory changed only by one of the characters above shows the same
       line split in its flag diff as on its card.
-- [ ] The existing control-character tests in `tests/test_ui.py` still pass.
+- [x] The existing control-character tests in `tests/test_ui.py` still pass.
 
 ## Notes
 
@@ -82,3 +82,17 @@ one shared `clean()` helper in `lore/ui.py` and, as its stated tradeoff, now
 deletes tabs inside flagged-publication diff lines too, so this item fixes
 both cards and the diff in one place: apply the tab rule inside `clean()`.
 The zero-width/direction-character question in the Notes stays out of scope.
+
+**Implementation (2026-11-13):**
+
+- `ui.clean` (`lore/ui.py`) expands tabs to spaces (`expandtabs(4)`) before
+  removing control characters, and also removes U+2028 and U+2029 (a new
+  `CARD_HIDDEN` table). `CONTROL_CHARACTERS`, which other modules share, is
+  unchanged. Every card field and every diff line goes through `clean`, so all of
+  them follow the same tab rule.
+- A new `store.text_lines()` splits only at `\n`, `\r\n` and a lone `\r`.
+  Both card bodies and `Store.flag_detail`'s diff use it, so the card and the
+  diff split lines the same way.
+- Tests are in `CardShapeTest` (`tests/test_ui.py`) and
+  `test_flag_detail_splits_lines_as_the_card_does` (`tests/test_store.py`).
+  The existing control-character tests still pass.

lore/store.py

@@ -432,6 +432,19 @@ def _clip(text: str) -> str:
     )
 
 
+def text_lines(text: str) -> list[str]:
+    """`text` split only where a line really ends: at `\n`, `\r\n` or a lone `\r`.
+    `str.splitlines` also breaks on form feeds, `\x1c`-`\x1e`, NEL and the Unicode
+    separators; a card and a flag diff print those as nothing, never as a new line
+    (CLI-015). A trailing line end adds no empty line, as with `splitlines`."""
+    if not text:
+        return []
+    lines = re.split(r"\r\n|\r|\n", text)
+    if not lines[-1]:
+        lines.pop()
+    return lines
+
+
 class Store:
     """Small SQLite repository for memories and Lore settings."""
 
@@ -1486,8 +1499,8 @@ def flag_detail(self, publication: Publication) -> list[dict[str, object]]:
                 # here means no reader has to guess which lines are headers.
                 diff = list(
                     difflib.unified_diff(
-                        snapshot["content"].splitlines(),
-                        memory["content"].splitlines(),
+                        text_lines(snapshot["content"]),
+                        text_lines(memory["content"]),
                         lineterm="",
                     )
                 )[2:]

lore/ui.py

@@ -5,9 +5,12 @@
 import textwrap
 from typing import Mapping, TextIO
 
-from .store import Memory, Publication
+from .store import Memory, Publication, text_lines
 
 CONTROL_CHARACTERS = dict.fromkeys((*range(32), *range(127, 160)))
+# What a card prints as nothing: the controls, and the Unicode line and paragraph
+# separators, which would otherwise start a new line in the terminal (CLI-015).
+CARD_HIDDEN = CONTROL_CHARACTERS | dict.fromkeys((0x2028, 0x2029))
 
 
 def clean(text: object) -> str:
@@ -16,8 +19,9 @@ def clean(text: object) -> str:
     Everything remote that reaches the terminal goes through here: card
     titles and bodies, a flag's actor and its diff lines (CLI-013). It is the
     one place to change when tabs are kept rather than deleted (CLI-015).
+    A tab is spacing, so it becomes spaces (CLI-015).
     """
-    return str(text).translate(CONTROL_CHARACTERS)
+    return str(text).expandtabs(4).translate(CARD_HIDDEN)
 
 
 def COLOR(
@@ -95,7 +99,7 @@ def memory_card(
     body = memory.content
     if len(body) > 1800:
         body = body[:1800].rstrip() + "\n…"
-    for paragraph in body.splitlines():
+    for paragraph in text_lines(body):
         paragraph = clean(paragraph)
         print(textwrap.fill(paragraph, width=78) if paragraph else "")
 
@@ -170,6 +174,6 @@ def publication_card(
         if text:
             print(paint("2", clean(f"{label}: {text}")))
     print()
-    for paragraph in publication.content.splitlines():
+    for paragraph in text_lines(publication.content):
         paragraph = clean(paragraph)
         print(textwrap.fill(paragraph, width=78) if paragraph else "")

tests/test_store.py

@@ -896,6 +896,25 @@ def test_flag_detail_reports_the_changed_memory_and_a_diff(self) -> None:
         self.assertIn("-Pricing lesson about deployment", diff_text)
         self.assertIn("+a changed body", diff_text)
 
+    def test_flag_detail_splits_lines_as_the_card_does(self) -> None:
+        """CLI-015: a separator that isn't a newline never splits a diff line."""
+        derived = self.publish()
+        with Store() as store:
+            before = store.search("Pricing lesson")[0].content
+            store.put(
+                source="test",
+                origin="native",
+                source_path="Pricing lesson",
+                source_key="Pricing lesson",
+                fingerprint="changed",
+                title="Pricing lesson",
+                content=before + "\x85tail",
+            )
+            publication = next(p for p in store.stale_publications() if p.id == derived)
+            (entry,) = store.flag_detail(publication)
+        changed = [line for line in entry["diff"] if line[:1] in "+-"]
+        self.assertEqual(changed, [f"-{before}", f"+{before}\x85tail"])
+
     def test_flag_detail_degrades_without_a_snapshot(self) -> None:
         # A publication approved before this feature shipped (or whose row
         # predates the publication_sources table) has no snapshot to diff

tests/test_ui.py

@@ -686,5 +686,65 @@ def test_eight_bit_control_characters_never_reach_the_terminal(self) -> None:
         )
 
 
+SEPARATORS = ("\x0b", "\x0c", "\x1c", "\x1d", "\x1e", "\x85", "\u2028", "\u2029")
+
+
+class CardShapeTest(PlainTerminalTest):
+    """CLI-015: a card looks like the text the owner approves. A tab is spacing, and
+    only a newline starts a line."""
+
+    def test_a_tab_is_spacing_in_every_field_of_both_cards(self) -> None:
+        with captured() as output:
+            ui.memory_card(_memory(title="t\tone", source="s\tsrc", content="b\tbody"))
+            ui.publication_card(
+                _publication(
+                    title="p\ttitle",
+                    topic="to\tpic",
+                    teaser="te\tase",
+                    useful_if="u\tif",
+                    not_useful_if="n\tif",
+                    sample="sa\tmple",
+                    content="c\tbody",
+                )
+            )
+        text = output.getvalue()
+        self.assertNotIn("\t", text)
+        for field in (
+            "t one",
+            "s src",
+            "b body",
+            "p title",
+            "to pic",
+            "te ase",
+            "u if",
+            "n if",
+            "sa mple",
+            "c body",
+        ):
+            first, second = field.split(" ")
+            with self.subTest(field=field):
+                self.assertRegex(text, rf"{first} +{second}")
+
+    def test_only_a_newline_starts_a_line_in_a_body(self) -> None:
+        body = "one" + "".join(f"{c}x{n}" for n, c in enumerate(SEPARATORS))
+        for card in (
+            lambda: ui.memory_card(_memory(content=body)),
+            lambda: ui.publication_card(_publication(content=body)),
+        ):
+            with self.subTest(card=card), captured() as output:
+                card()
+            text = output.getvalue()
+            for character in SEPARATORS:
+                self.assertNotIn(character, text)
+            self.assertEqual(text.splitlines()[-1], "onex0x1x2x3x4x5x6x7")
+
+    def test_a_carriage_return_joins_a_title_and_breaks_a_body(self) -> None:
+        with captured() as output:
+            ui.memory_card(_memory(title="a\rb", content="c\rd"))
+        lines = output.getvalue().splitlines()
+        self.assertIn("ab", lines)
+        self.assertEqual(lines[-2:], ["c", "d"])
+
+
 if __name__ == "__main__":
     unittest.main()