trillian_log_api.proto

// Copyright 2016 Google LLC. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto3";

package trillian;

option go_package = "github.com/google/trillian";
option java_multiple_files = true;
option java_outer_classname = "TrillianLogApiProto";
option java_package = "com.google.trillian.proto";

import "google/protobuf/timestamp.proto";
import "google/rpc/status.proto";
import "trillian.proto";

// The TrillianLog service provides access to an append-only Log data structure
// as described in the [Verifiable Data
// Structures](docs/papers/VerifiableDataStructures.pdf) paper.
//
// The API supports adding new entries to the Merkle tree for a specific Log
// instance (identified by its log_id) in two modes:
//  - For a normal log, new leaf entries are queued up for subsequent
//    inclusion in the log, and the leaves are assigned consecutive leaf_index
//    values as part of that integration process.
//  - For a 'pre-ordered log', new entries have an already-defined leaf
//    ordering, and leaves are only integrated into the Merkle tree when a
//    contiguous range of leaves is available.
//
// The API also supports read operations to retrieve leaf contents, and to
// provide cryptographic proofs of leaf inclusion and of the append-only nature
// of the Log.
//
// Each API request also includes a charge_to field, which allows API users
// to provide quota identifiers that should be "charged" for each API request
// (and potentially rejected with codes.ResourceExhausted).
//
// Various operations on the API also allows for 'server skew', which can occur
// when different API requests happen to be handled by different server instances
// that may not all be up to date.  An API request that is relative to a specific
// tree size may reach a server instance that is not yet aware of this tree size;
// in this case the server will typically return an OK response that contains:
//  - a signed log root that indicates the tree size that it is aware of
//  - an empty response otherwise.
service TrillianLog {
  // QueueLeaf adds a single leaf to the queue of pending leaves for a normal
  // log.
  rpc QueueLeaf(QueueLeafRequest) returns (QueueLeafResponse) {}

  // GetInclusionProof returns an inclusion proof for a leaf with a given index
  // in a particular tree.
  //
  // If the requested tree_size is larger than the server is aware of, the
  // response will include the latest known log root and an empty proof.
  rpc GetInclusionProof(GetInclusionProofRequest)
      returns (GetInclusionProofResponse) {}

  // GetInclusionProofByHash returns an inclusion proof for any leaves that have
  // the given Merkle hash in a particular tree.
  //
  // If any of the leaves that match the given Merkle has have a leaf index that
  // is beyond the requested tree size, the corresponding proof entry will be empty.
  rpc GetInclusionProofByHash(GetInclusionProofByHashRequest)
      returns (GetInclusionProofByHashResponse) {}

  // GetConsistencyProof returns a consistency proof between different sizes of
  // a particular tree.
  //
  // If the requested tree size is larger than the server is aware of,
  // the response will include the latest known log root and an empty proof.
  rpc GetConsistencyProof(GetConsistencyProofRequest)
      returns (GetConsistencyProofResponse) {}

  // GetLatestSignedLogRoot returns the latest log root for a given tree,
  // and optionally also includes a consistency proof from an earlier tree size
  // to the new size of the tree.
  //
  // If the earlier tree size is larger than the server is aware of,
  // an InvalidArgument error is returned.
  rpc GetLatestSignedLogRoot(GetLatestSignedLogRootRequest)
      returns (GetLatestSignedLogRootResponse) {}

  // GetEntryAndProof returns a log leaf and the corresponding inclusion proof
  // to a specified tree size, for a given leaf index in a particular tree.
  //
  // If the requested tree size is unavailable but the leaf is
  // in scope for the current tree, the returned proof will be for the
  // current tree size rather than the requested tree size.
  rpc GetEntryAndProof(GetEntryAndProofRequest)
      returns (GetEntryAndProofResponse) {}

  // InitLog initializes a particular tree, creating the initial signed log
  // root (which will be of size 0).
  rpc InitLog(InitLogRequest) returns (InitLogResponse) {}


  // AddSequencedLeaves adds a batch of leaves with assigned sequence numbers
  // to a pre-ordered log.  The indices of the provided leaves must be contiguous.
  rpc AddSequencedLeaves(AddSequencedLeavesRequest)
      returns (AddSequencedLeavesResponse) {}

  // GetLeavesByRange returns a batch of leaves whose leaf indices are in a
  // sequential range.
  rpc GetLeavesByRange(GetLeavesByRangeRequest)
      returns (GetLeavesByRangeResponse) {}
}

// ChargeTo describes the user(s) associated with the request whose quota should
// be checked and charged.
message ChargeTo {
  // user is a list of personality-defined strings.
  // Trillian will treat them as /User/%{user}/... keys when checking and
  // charging quota.
  // If one or more of the specified users has insufficient quota, the
  // request will be denied.
  //
  // As an example, a Certificate Transparency frontend might set the following
  // user strings when sending a QueueLeaf request to the Trillian log:
  //   - The requesting IP address.
  //     This would limit the number of requests per IP.
  //   - The "intermediate-<hash>" for each of the intermediate certificates in
  //     the submitted chain.
  //     This would have the effect of limiting the rate of submissions under
  //     a given intermediate/root.
  repeated string user = 1;
}

message QueueLeafRequest {
  int64 log_id = 1;
  LogLeaf leaf = 2;
  ChargeTo charge_to = 3;
}

message QueueLeafResponse {
  // queued_leaf describes the leaf which is or will be incorporated into the
  // Log.  If the submitted leaf was already present in the Log (as indicated by
  // its leaf identity hash), then the returned leaf will be the pre-existing
  // leaf entry rather than the submitted leaf.
  QueuedLogLeaf queued_leaf = 2;
}

message GetInclusionProofRequest {
  int64 log_id = 1;
  int64 leaf_index = 2;
  int64 tree_size = 3;
  ChargeTo charge_to = 4;
}

message GetInclusionProofResponse {
  // The proof field may be empty if the requested tree_size was larger
  // than that available at the server (e.g. because there is skew between
  // server instances, and an earlier client request was processed by a
  // more up-to-date instance).  In this case, the signed_log_root
  // field will indicate the tree size that the server is aware of, and
  // the proof field will be empty.
  Proof proof = 2;
  SignedLogRoot signed_log_root = 3;
}

message GetInclusionProofByHashRequest {
  int64 log_id = 1;
  // The leaf hash field provides the Merkle tree hash of the leaf entry
  // to be retrieved.
  bytes leaf_hash = 2;
  int64 tree_size = 3;
  bool order_by_sequence = 4;
  ChargeTo charge_to = 5;
}

message GetInclusionProofByHashResponse {
  // Logs can potentially contain leaves with duplicate hashes so it's possible
  // for this to return multiple proofs.  If the leaf index for a particular
  // instance of the requested Merkle leaf hash is beyond the requested tree
  // size, the corresponding proof entry will be missing.
  repeated Proof proof = 2;
  SignedLogRoot signed_log_root = 3;
}

message GetConsistencyProofRequest {
  int64 log_id = 1;
  int64 first_tree_size = 2;
  int64 second_tree_size = 3;
  ChargeTo charge_to = 4;
}

message GetConsistencyProofResponse {
  // The proof field may be empty if the requested tree_size was larger
  // than that available at the server (e.g. because there is skew between
  // server instances, and an earlier client request was processed by a
  // more up-to-date instance).  In this case, the signed_log_root
  // field will indicate the tree size that the server is aware of, and
  // the proof field will be empty.
  Proof proof = 2;
  SignedLogRoot signed_log_root = 3;
}

message GetLatestSignedLogRootRequest {
  int64 log_id = 1;
  ChargeTo charge_to = 2;
  // If first_tree_size is non-zero, the response will include a consistency
  // proof between first_tree_size and the new tree size (if not smaller).
  int64 first_tree_size = 3;
}

message GetLatestSignedLogRootResponse {
  SignedLogRoot signed_log_root = 2;
  // proof is filled in with a consistency proof if first_tree_size in
  // GetLatestSignedLogRootRequest is non-zero (and within the tree size
  // available at the server).
  Proof proof = 3;
}

message GetEntryAndProofRequest {
  int64 log_id = 1;
  int64 leaf_index = 2;
  int64 tree_size = 3;
  ChargeTo charge_to = 4;
}

message GetEntryAndProofResponse {
  Proof proof = 2;
  LogLeaf leaf = 3;
  SignedLogRoot signed_log_root = 4;
}

message InitLogRequest {
  int64 log_id = 1;
  ChargeTo charge_to = 2;
}

message InitLogResponse {
  SignedLogRoot created = 1;
}

message AddSequencedLeavesRequest {
  int64 log_id = 1;
  repeated LogLeaf leaves = 2;
  ChargeTo charge_to = 4;
}

message AddSequencedLeavesResponse {
  // Same number and order as in the corresponding request.
  repeated QueuedLogLeaf results = 2;
}

message GetLeavesByRangeRequest {
  int64 log_id = 1;
  int64 start_index = 2;
  int64 count = 3;
  ChargeTo charge_to = 4;
}

message GetLeavesByRangeResponse {
  // Returned log leaves starting from the `start_index` of the request, in
  // order. There may be fewer than `request.count` leaves returned, if the
  // requested range extended beyond the size of the tree or if the server opted
  // to return fewer leaves than requested.
  repeated LogLeaf leaves = 1;
  SignedLogRoot signed_log_root = 2;
}

// QueuedLogLeaf provides the result of submitting an entry to the log.
// TODO(pavelkalinnikov): Consider renaming it to AddLogLeafResult or the like.
message QueuedLogLeaf {
  // The leaf as it was stored by Trillian. Empty unless `status.code` is:
  //  - `google.rpc.OK`: the `leaf` data is the same as in the request.
  //  - `google.rpc.ALREADY_EXISTS` or 'google.rpc.FAILED_PRECONDITION`: the
  //    `leaf` is the conflicting one already in the log.
  LogLeaf leaf = 1;

  // The status of adding the leaf.
  //  - `google.rpc.OK`: successfully added.
  //  - `google.rpc.ALREADY_EXISTS`: the leaf is a duplicate of an already
  //    existing one. Either `leaf_identity_hash` is the same in the `LOG`
  //    mode, or `leaf_index` in the `PREORDERED_LOG`.
  //  - `google.rpc.FAILED_PRECONDITION`: A conflicting entry is already
  //    present in the log, e.g., same `leaf_index` but different `leaf_data`.
  google.rpc.Status status = 2;
}

// LogLeaf describes a leaf in the Log's Merkle tree, corresponding to a single log entry.
// Each leaf has a unique leaf index in the scope of this tree.  Clients submitting new
// leaf entries should only set the following fields:
//   - leaf_value
//   - extra_data (optionally)
//   - leaf_identity_hash (optionally)
//   - leaf_index (iff the log is a PREORDERED_LOG)
message LogLeaf {
  // merkle_leaf_hash holds the Merkle leaf hash over leaf_value.  This is
  // calculated by the Trillian server when leaves are added to the tree, using
  // the defined hashing algorithm and strategy for the tree; as such, the client
  // does not need to set it on leaf submissions.
  bytes merkle_leaf_hash = 1;

  // leaf_value holds the data that forms the value of the Merkle tree leaf.
  // The client should set this field on all leaf submissions, and is
  // responsible for ensuring its validity (the Trillian server treats it as an
  // opaque blob).
  bytes leaf_value = 2;

  // extra_data holds additional data associated with the Merkle tree leaf.
  // The client may set this data on leaf submissions, and the Trillian server
  // will return it on subsequent read operations. However, the contents of
  // this field are not covered by and do not affect the Merkle tree hash
  // calculations.
  bytes extra_data = 3;

  // leaf_index indicates the index of this leaf in the Merkle tree.
  // This field is returned on all read operations, but should only be
  // set for leaf submissions in PREORDERED_LOG mode (for a normal log
  // the leaf index is assigned by Trillian when the submitted leaf is
  // integrated into the Merkle tree).
  int64 leaf_index = 4;

  // leaf_identity_hash provides a hash value that indicates the client's
  // concept of which leaf entries should be considered identical.
  //
  // This mechanism allows the client personality to indicate that two leaves
  // should be considered "duplicates" even though their `leaf_value`s differ.
  //
  // If this is not set on leaf submissions, the Trillian server will take its
  // value to be the same as merkle_leaf_hash (and thus only leaves with
  // identical leaf_value contents will be considered identical).
  //
  // For example, in Certificate Transparency each certificate submission is
  // associated with a submission timestamp, but subsequent submissions of the
  // same certificate should be considered identical.  This is achieved
  // by setting the leaf identity hash to a hash over (just) the certificate,
  // whereas the Merkle leaf hash encompasses both the certificate and its
  // submission time -- allowing duplicate certificates to be detected.
  //
  //
  // Continuing the CT example, for a CT mirror personality (which must allow
  // dupes since the source log could contain them), the part of the
  // personality which fetches and submits the entries might set
  // `leaf_identity_hash` to `H(leaf_index||cert)`.
  //
  // TODO(pavelkalinnikov): Consider instead using `H(cert)` and allowing
  // identity hash dupes in `PREORDERED_LOG` mode, for it can later be
  // upgraded to `LOG` which will need to correctly detect duplicates with
  // older entries when new ones get queued.
  bytes leaf_identity_hash = 5;

  // queue_timestamp holds the time at which this leaf was queued for
  // inclusion in the Log, or zero if the entry was submitted without
  // queuing. Clients should not set this field on submissions.
  google.protobuf.Timestamp queue_timestamp = 6;

  // integrate_timestamp holds the time at which this leaf was integrated into
  // the tree.  Clients should not set this field on submissions.
  google.protobuf.Timestamp integrate_timestamp = 7;
}