Add README to sync's components/ directories pointing to new docs
crrev.com/c/3401841 moves the docs/sync/ content to docs/website/
(different repository). Indicate this in the README-s and delete
docs/sync/.
Bug: 1006699
Fixed: 1101716
Change-Id: I9096664d9bc1064a33e1b4cfdaa8adc2bdab9eb9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3404277
Reviewed-by: Marc Treib <[email protected]>
Commit-Queue: Victor Vianna <[email protected]>
Auto-Submit: Victor Vianna <[email protected]>
Cr-Commit-Position: refs/heads/main@{#964023}
diff --git a/docs/README.md b/docs/README.md
index 16bf8f2d..4eee5a2a 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -144,10 +144,8 @@
renderer to the screen.
* [Using the Origin Trials Framework](origin_trials_integration.md) - A
framework for conditionally enabling experimental APIs for testing.
-* [`ClientTagBasedModelTypeProcessor` in Unified Sync and Storage](sync/uss/client_tag_based_model_type_processor.md) -
- Notes on the central data structure used in Chrome Sync.
-* [Chrome Sync's Model API](sync/model_api.md) - Data models used for syncing
- information across devices using Chrome Sync.
+* [Chrome Sync](https://source.chromium.org/chromium/chromium/src/+/main:docs/website/site/developers/design-documents/sync) -
+ Docs for the subsystem that allows one to sync data across devices.
* [Ozone Overview](ozone_overview.md) - Ozone is an abstraction layer between
the window system and low level input and graphics.
* [Optimizing Chrome Web UIs](optimizing_web_uis.md) - Notes on making webuis
diff --git a/docs/sync/DIR_METADATA b/docs/sync/DIR_METADATA
deleted file mode 100644
index 6ae00c05..0000000
--- a/docs/sync/DIR_METADATA
+++ /dev/null
@@ -1,12 +0,0 @@
-# Metadata information for this directory.
-#
-# For more information on DIR_METADATA files, see:
-# https://source.chromium.org/chromium/infra/infra/+/main:go/src/infra/tools/dirmd/README.md
-#
-# For the schema of this file, see Metadata message:
-# https://source.chromium.org/chromium/infra/infra/+/main:go/src/infra/tools/dirmd/proto/dir_metadata.proto
-
-monorail {
- component: "Services>Sync"
-}
-team_email: "[email protected]"
\ No newline at end of file
diff --git a/docs/sync/model_api.md b/docs/sync/model_api.md
deleted file mode 100644
index 7905ccf..0000000
--- a/docs/sync/model_api.md
+++ /dev/null
@@ -1,302 +0,0 @@
-# Chrome Sync's Model API
-
-Chrome Sync operates on discrete, explicitly defined model types (bookmarks,
-preferences, tabs, etc). These model types are individually responsible for
-implementing their own local storage and responding to remote changes. This
-guide is for developers interested in syncing data for their model type to the
-cloud using Chrome Sync. It describes the newest version of the API, known as
-Unified Sync and Storage (USS). There is also the deprecated [SyncableService
-API] (aka Directory), which as of mid-2019 is still used by several legacy model
-types, but "wrapped into" USS (see [SyncableServiceBasedBridge]).
-
-[SyncableService API]: https://www.chromium.org/developers/design-documents/sync/syncable-service-api
-[SyncableServiceBasedBridge]: https://cs.chromium.org/chromium/src/components/sync/model/syncable_service_based_bridge.h
-
-[TOC]
-
-## Overview
-
-To correctly sync data, USS requires that sync metadata be stored alongside your
-model data in a way such that they are written together atomically. **This is
-very important!** Sync must be able to update the metadata for any local data
-changes as part of the same write to disk. If you attempt to write data to disk
-and only notify sync afterwards, a crash in between the two writes can result in
-changes being dropped and never synced to the server, or data being duplicated
-due to being committed more than once.
-
-[`ModelTypeSyncBridge`][Bridge] is the interface the model code must implement.
-The bridge is usually owned by a [`KeyedService`][KeyedService].
-The correct place for the bridge generally lies as close to where your model
-data is stored as possible, as the bridge needs to be able to inject metadata
-updates into any local data changes that occur.
-
-The bridge owns a [`ModelTypeChangeProcessor`][MTCP] object, which it uses to
-communicate local changes to sync using the `Put` and `Delete` methods.
-The processor will communicate remote changes from sync to the bridge using the
-`MergeSyncData` and `ApplySyncChanges` methods, respectively for the initial
-merge of remote and local data, and for incremental changes coming from sync.
-[`MetadataChangeList`][MCL] is the way sync communicates metadata changes to the
-storage mechanism. Note that it is typically implemented on a per-storage basis,
-not a per-type basis.
-
-[Bridge]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_sync_bridge.h
-[KeyedService]: https://cs.chromium.org/chromium/src/components/keyed_service/core/keyed_service.h
-[MTCP]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_change_processor.h
-[MCL]: https://cs.chromium.org/chromium/src/components/sync/model/metadata_change_list.h
-
-## Data
-
-### Specifics
-
-Model types will define a proto that contains the necessary fields of the
-corresponding native type (e.g. [`TypedUrlSpecifics`][TypedUrlSpecifics]
-contains a URL and a list of visit timestamps) and include it as a field in the
-generic [`EntitySpecifics`][EntitySpecifics] proto. This is the form that all
-communications with sync will use. This proto form of the model data is referred
-to as the specifics.
-
-[TypedUrlSpecifics]: https://cs.chromium.org/chromium/src/components/sync/protocol/typed_url_specifics.proto
-[EntitySpecifics]: https://cs.chromium.org/search/?q="message+EntitySpecifics"+file:sync.proto
-
-### Identifiers
-
-There are two primary identifiers for entities: **storage key** and **client
-tag**. The bridge will need to take an [`EntityData`][EntityData] object (which
-contains the specifics) and be able generate both of these from it. For
-non-legacy types without significant performance concerns, these will generally
-be the same.
-
-The storage key is meant to be the primary key in the local model/database.
-It’s what’s used to refer to entities most of the time and, as its name implies,
-the bridge needs to be able to look up local data and metadata entries in the
-store using it. Because it is a local identifier, it can change as part of
-database migrations, etc. This may be desirable for efficiency reasons.
-
-The client tag is used to generate the **client tag hash**, which will identify
-entities **across clients**. This means that its implementation can **never
-change** once entities have begun to sync, without risking massive duplication
-of entities. This means it must be generated using only immutable data in the
-specifics. If your type does not have any immutable fields to use, you will need
-to add one (e.g. a GUID, though be wary as they have the potential to conflict).
-While the hash gets written to disk as part of the metadata, the tag itself is
-never persisted locally.
-
-[EntityData]: https://cs.chromium.org/chromium/src/components/sync/protocol/entity_data.h
-
-## Storage
-
-A crucial requirement of USS is that the model must add support for keeping
-sync’s metadata in the same storage as its normal data. The metadata consists of
-one [`EntityMetadata`][EntityMetadata] proto for each data entity, and one
-[`ModelTypeState`][ModelTypeState] proto containing metadata pertaining to the
-state of the entire type (the progress marker, for example). This typically
-requires two extra tables in a database to do (one for each type of proto).
-
-Since the processor doesn’t know anything about the store, the bridge provides
-it with an implementation of the [`MetadataChangeList`][MCL] interface. The
-change processor writes metadata through this interface when changes occur, and
-the bridge simply has to ensure it gets passed along to the store and written
-along with the data changes.
-
-[EntityMetadata]: https://cs.chromium.org/chromium/src/components/sync/protocol/entity_metadata.proto
-[ModelTypeState]: https://cs.chromium.org/chromium/src/components/sync/protocol/model_type_state.proto
-
-### ModelTypeStore
-
-While the model type may store its data however it chooses, many types use
-[`ModelTypeStore`][Store], which was created specifically to provide a
-convenient persistence solution. It’s backed by a [LevelDB] to store serialized
-protos to disk. `ModelTypeStore` provides two `MetadataChangeList`
-implementations for convenience; both accessed via
-[`ModelTypeStore::WriteBatch`][WriteBatch]. One passes metadata changes directly
-into an existing `WriteBatch` and another caches them in memory until a
-`WriteBatch` exists to consume them.
-
-The store interface abstracts away the type and will handle setting up tables
-for the type’s data, so multiple `ModelTypeStore` objects for different types
-can share the same LevelDB backend just by specifying the same path and task
-runner. Sync already has a backend it uses for DeviceInfo that can be shared by
-other types via the [`ModelTypeStoreService`][StoreService].
-
-[Store]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_store.h
-[LevelDB]: https://github.com/google/leveldb/blob/master/doc/index.md
-[WriteBatch]: https://cs.chromium.org/search/?q="class+WriteBatch"+file:model_type_store_base.h
-[StoreService]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_store_service.h
-
-## Implementing ModelTypeSyncBridge
-
-### Initialization
-
-The bridge is required to load all of the metadata for its type from storage and
-provide it to the processor via the [`ModelReadyToSync`][ModelReadyToSync]
-method **before any local changes occur**. This can be tricky if the thread the
-bridge runs on is different from the storage mechanism. No data will be synced
-with the server if the processor is never informed that the model is ready.
-
-Since the tracking of changes and updating of metadata is completely
-independent, there is no need to wait for the sync engine to start before
-changes can be made. This prevents the need for an expensive association step in
-the initialization.
-
-[ModelReadyToSync]: https://cs.chromium.org/search/?q=ModelReadyToSync+file:/model_type_change_processor.h
-
-### MergeSyncData
-
-This method is called only once, when a type is first enabled. Sync will
-download all the data it has for the type from the server and provide it to the
-bridge using this method. Sync filters out any tombstones for this call, so
-`EntityData::is_deleted()` will never be true for the provided entities. The
-bridge must then examine the sync data and the local data and merge them
-together:
-
-* Any remote entities that don’t exist locally must be be written to local
- storage.
-* Any local entities that don’t exist remotely must be provided to sync via
- [`ModelTypeChangeProcessor::Put`][Put].
-* Any entities that appear in both sets must be merged and the model and sync
- informed accordingly. Decide which copy of the data to use (or a merged
- version or neither) and update the local store and sync as necessary to
- reflect the decision. How the decision is made can vary by model type.
-
-The [`MetadataChangeList`][MCL] passed into the function is already populated
-with metadata for all the data passed in (note that neither the data nor the
-metadata have been committed to storage yet at this point). It must be given to
-the processor for any `Put` or `Delete` calls so the relevant metadata can be
-added/updated/deleted, and then passed to the store for persisting along with
-the data.
-
-Note that if sync gets disabled and the metadata cleared, entities that
-originated from other clients will exist as “local” entities the next time sync
-starts and merge is called. Since tombstones are not provided for merge, this
-can result in reviving the entity if it had been deleted on another client in
-the meantime.
-
-[Put]: https://cs.chromium.org/search/?q=Put+file:/model_type_change_processor.h
-
-### ApplySyncChanges
-
-While `MergeSyncData` provides the state of sync data using `EntityData`
-objects, `ApplySyncChanges` provides changes to the state using
-[`EntityChange`][EntityChange] objects. These changes must be applied to the
-local state.
-
-Here’s an example implementation of a type using `ModelTypeStore`:
-
-```cpp
-absl::optional<ModelError> DeviceInfoSyncBridge::ApplySyncChanges(
- std::unique_ptr<MetadataChangeList> metadata_change_list,
- EntityChangeList entity_changes) {
- std::unique_ptr<WriteBatch> batch = store_->CreateWriteBatch();
- for (const EntityChange& change : entity_changes) {
- if (change.type() == EntityChange::ACTION_DELETE) {
- batch->DeleteData(change.storage_key());
- } else {
- batch->WriteData(change.storage_key(),
- change.data().specifics.your_type().SerializeAsString());
- }
- }
-
- batch->TakeMetadataChangesFrom(std::move(metadata_change_list));
- store_->CommitWriteBatch(std::move(batch), base::BindOnce(...));
- NotifyModelOfChanges();
- return {};
-}
-```
-
-A conflict can occur when an entity has a pending local commit when an update
-for the same entity comes from another client. In this case, the bridge’s
-[`ResolveConflict`][ResolveConflict] method will have been called prior to the
-`ApplySyncChanges` call in order to determine what should happen. This method
-defaults to having the remote version overwrite the local version unless the
-remote version is a tombstone, in which case the local version wins.
-
-[EntityChange]: https://cs.chromium.org/chromium/src/components/sync/model/entity_change.h
-[ResolveConflict]: https://cs.chromium.org/search/?q=ResolveConflict+file:/model_type_sync_bridge.h
-
-### Local changes
-
-The [`ModelTypeChangeProcessor`][MTCP] must be informed of any local changes via
-its `Put` and `Delete` methods. Since the processor cannot do any useful
-metadata tracking until `MergeSyncData` is called, the `IsTrackingMetadata`
-method is provided. It can be checked as an optimization to prevent unnecessary
-processing preparing the parameters to a `Put` or `Delete` call.
-
-Here’s an example of handling a local write using `ModelTypeStore`:
-
-```cpp
-void WriteLocalChange(std::string key, ModelData data) {
- std::unique_ptr<WriteBatch> batch = store_->CreateWriteBatch();
- if (change_processor()->IsTrackingMetadata()) {
- change_processor()->Put(key, ModelToEntityData(data),
- batch->GetMetadataChangeList());
- }
- batch->WriteData(key, data.specifics->SerializeAsString());
- store_->CommitWriteBatch(std::move(batch), base::BindOnce(...));
-}
-```
-
-## Error handling
-
-If any errors occur during store operations that could compromise the
-consistency of the data and metadata, the processor’s
-[`ReportError`][ReportError] method should be called. The only exception to this
-is errors during `MergeSyncData` or `ApplySyncChanges`, which should just return
-a [`ModelError`][ModelError].
-
-This will inform sync of the error, which will stop all communications with the
-server so bad data doesn’t get synced. Since the metadata might no longer be
-valid, the bridge will asynchronously receive an `ApplyStopSyncChanges` call
-with a non-null `MetadataChangeList` parameter. All the metadata will be cleared
-from the store (if possible), and the type will be started again from scratch on
-the next client restart.
-
-[ReportError]: https://cs.chromium.org/search/?q=ReportError+file:/model_type_change_processor.h
-[ModelError]: https://cs.chromium.org/chromium/src/components/sync/model/model_error.h
-
-## Sync Integration Checklist
-
-* Define your specifics proto in [`//components/sync/protocol/`][protocol].
-* Add a field for it to [`EntitySpecifics`][EntitySpecifics].
-* Add it to the [`ModelType`][ModelType] enum and
- [`kModelTypeInfoMap`][info_map].
-* Add it to the [proto value conversions][conversions] files.
-* Register a [`ModelTypeController`][ModelTypeController] for your type in
- [`SyncApiComponentFactoryImpl::CreateCommonDataTypeControllers`][CreateCommonDataTypeControllers] or platform-specific equivalent in
- [`ChromeSyncClient::CreateDataTypeControllers`][CreateDataTypeControllers].
-* Add your KeyedService dependency to
- [`SyncServiceFactory`][SyncServiceFactory].
-* Add an field for encrypted data to [`NigoriSpecifics`][NigoriSpecifics].
-* If your type should have its own toggle in sync settings, add an entry to
- the [`UserSelectableType`][UserSelectableType] enum, add a
- [preference][pref_names] for tracking whether your type is enabled, and
- map your type to the pref in [`GetPrefNameForType`][GetPrefName].
-* Otherwise, if your type should be included in an existing toggle in sync
- settings, add it in [`GetUserSelectableTypeInfo`]
- [GetUserSelectableTypeInfo].
-* Add to the `SyncModelTypes` enum in [`enums.xml`][enums] and to the
- `SyncModelType` suffix in [`histograms.xml`][histograms].
-
-[protocol]: https://cs.chromium.org/chromium/src/components/sync/protocol/
-[ModelType]: https://cs.chromium.org/chromium/src/components/sync/base/model_type.h
-[info_map]: https://cs.chromium.org/search/?q="kModelTypeInfoMap%5B%5D"+file:model_type.cc
-[conversions]: https://cs.chromium.org/chromium/src/components/sync/protocol/proto_value_conversions.h
-[ModelTypeController]: https://cs.chromium.org/chromium/src/components/sync/driver/model_type_controller.h
-[CreateCommonDataTypeControllers]: https://cs.chromium.org/search/?q="SyncApiComponentFactoryImpl::CreateCommonDataTypeControllers"
-[CreateDataTypeControllers]: https://cs.chromium.org/search/?q="ChromeSyncClient::CreateDataTypeControllers"
-[SyncServiceFactory]: https://cs.chromium.org/search/?q=:SyncServiceFactory%5C(%5C)
-[NigoriSpecifics]: https://cs.chromium.org/chromium/src/components/sync/protocol/nigori_specifics.proto
-[UserSelectableType]: https://cs.chromium.org/chromium/src/components/sync/base/user_selectable_type.h?type=cs&q="enum+class+UserSelectableType"
-[pref_names]: https://cs.chromium.org/chromium/src/components/sync/base/pref_names.h
-[GetPrefName]: https://cs.chromium.org/search/?q=GetPrefNameForType+file:sync_prefs.cc
-[GetUserSelectableTypeInfo]: https://cs.chromium.org/chromium/src/components/sync/base/user_selectable_type.cc?type=cs&q="UserSelectableTypeInfo+GetUserSelectableTypeInfo"+f:components/sync/base/user_selectable_type.cc
-[enums]: https://cs.chromium.org/chromium/src/tools/metrics/histograms/enums.xml
-[histograms]: https://cs.chromium.org/chromium/src/tools/metrics/histograms/histograms.xml
-[DataTypeHistogram]: https://cs.chromium.org/chromium/src/components/sync/base/data_type_histogram.h
-
-## Testing
-
-The [`TwoClientTypedUrlsSyncTest`][UssTest] suite is probably a good place to start
-for integration testing. Especially note the use of a `StatusChangeChecker` to
-wait for events to happen.
-
-[UssTest]: https://cs.chromium.org/chromium/src/chrome/browser/sync/test/integration/two_client_typed_urls_sync_test.cc
diff --git a/docs/sync/uss/client_tag_based_model_type_processor.md b/docs/sync/uss/client_tag_based_model_type_processor.md
deleted file mode 100644
index 5c0b6d5..0000000
--- a/docs/sync/uss/client_tag_based_model_type_processor.md
+++ /dev/null
@@ -1,120 +0,0 @@
-# ClientTagBasedModelTypeProcessor
-
-The [`ClientTagBasedModelTypeProcessor`][SMTP] is a crucial piece of the USS
-codepath. It lives on the model thread and performs the tracking of sync
-metadata for the [`ModelTypeSyncBridge`][MTSB] that owns it by implementing the
-[`ModelTypeChangeProcessor`][MTCP] interface, as well as sending commit requests
-to the [`ModelTypeWorker`][MTW] on the sync thread via the [`CommitQueue`][CQ]
-interface and receiving updates from the same worker via the
-[`ModelTypeProcessor`][MTP] interface.
-
-This processor supports types that use a client tag, which is currently
-includes all except bookmarks. This means all changes in flight (either incoming
-remote changes provided via the [`ModelTypeWorker`][MTW], or local changes
-reported by the [`ModelTypeSyncBridge`][MTSB]) must specify a client tag, which
-is considered (after being hashed) the main global identifier of a sync entity.
-
-[SMTP]: https://cs.chromium.org/chromium/src/components/sync/model/client_tag_based_model_type_processor.h
-[MTSB]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_sync_bridge.h
-[MTCP]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_change_processor.h
-[MTW]: https://cs.chromium.org/chromium/src/components/sync/engine/model_type_worker.h
-[CQ]: https://cs.chromium.org/chromium/src/components/sync/engine/commit_queue.h
-[MTP]: https://cs.chromium.org/chromium/src/components/sync/engine/model_type_processor.h
-
-[TOC]
-
-## Lifetime
-
-The bridge owns a processor object at all times and operates on the same thread
-as it. If sync is disabled, the processor is destroyed but a new one is
-immediately created to replace it.
-
-## Processor State Machines
-
-The processor sits between the model bridge and the sync engine. It has
-knowledge of what state each is in based on the calls it has received and
-performed. The states are not stored explicitly, but are implicit based on
-state stored in the processor. Here are the states of each, with notes on their
-transitions and how to determine them.
-
-### Model States
-
-* `UNREADY`
- * Waiting for `ModelReadyToStart` to be called.
- * Determined by: `waiting_for_metadata_ && !model_error_`
-* `NEEDS_DATA`
- * Waiting for data for pending commits to be loaded.
- * This state is skipped if there are no pending commits.
- * Determined by: `waiting_for_pending_data_ && !model_error_`
-* `READY`
- * The model is completely ready to sync.
- * Determined by: `!waiting_for_metadata_ && !waiting_for_pending_data &&
- !model_error`
-* `ERROR`
- * Something in the model or storage broke.
- * This state is permanent until DisableSync destroys the object.
- * Determined by: `!!model_error_`
-
-### Sync States
-
-* `DISCONNECTED`
- * Sync for this type has not started.
- * This state can be re-entered from any other state if Disconnect is
- called.
- * Determined by: `!error_handler_`.
-* `STARTED`
- * Sync has started but the model is not yet `READY` (or `ERROR`).
- * This state is skipped if the model is ready before sync is.
- * Determined by: `error_handler_ && start_callback_`
-* `CONNECT_PENDING`
- * Both the model and sync are ready. The start callback has been called
- and we're waiting to connect to the sync thread.
- * If the model was `ERROR`, the error is passed along and the callback is
- cleared; we're really waiting for DisableSync instead of connect.
- * Determined by: `error_handler_ && !start_callback_`
-* `CONNECTED`
- * We have a [`CommitQueue`][CQ] that passes changes to the
- [`ModelTypeWorker`][MTW] on the sync thread.
- * Determined by: `!!worker_`
-
-### Processor States
-
-Based on the interplay of the model and sync states, the processor effectively
-progresses through 3 states worth noting:
-
-* `UNINITIALIZED`
- * Metadata isn't loaded so we have no knowledge of entities.
- * `Put` and `Delete` calls are not allowed in this state (will DCHECK).
-* `NOT_TRACKING`
- * Indicates that not metadata is being tracked and that `Put` and `Delete`
- calls will be ignored.
- * This state is entered if the loaded metadata shows an initial merge
- hasn't happened (`ModelTypeState::initial_sync_done` is false).
- * Exposed via `IsTrackingMetadata` for optimization, not correctness.
-* `TRACKING`
- * Indicates that metadata is being tracked and `Put` and `Delete` calls
- must happen for entity changes.
- * This state is entered if the loaded metadata shows an initial merge
- has happened (`ModelTypeState::initial_sync_done` is true).
-* `SYNCING`
- * Indicates that commits can be sent and updates can be received from the
- sync server. This is a superstate of `TRACKING`.
- * If the processor was in `TRACKING`, it progresses to this state as soon
- as it gets connected to the worker.
- * If the processor was in `NOT_TRACKING`, it progresses to this state
- after `MergeSyncData` is called and the metadata is initialized.
-
-## Entity Tracker
-
-The [`ProcessorEntity`][PET] tracks the state of individual entities for
-the processor. It keeps the [`EntityMetadata`][EM] proto in memory, as well as
-any pending commit data until it gets acked by the server. It also stores the
-special `commit_requested_sequence_number_`, which tracks the sequence number of
-the last version that's been sent to the server.
-
-The tracker holds the metadata in memory forever, which is needed so we know
-what to update the on-disk memory with when we get a new local or remote change.
-Changing this would require being able to handle updates asynchronously.
-
-[PET]: https://cs.chromium.org/chromium/src/components/sync/model/processor_entity.h
-[EM]: https://cs.chromium.org/chromium/src/components/sync/protocol/entity_metadata.proto
