Changes announced December 4, 2024

We are planning to modify the Protobuf debug APIs (including Protobuf AbslStringify, proto2::ShortFormat, proto2::Utf8Format, Message::DebugString, Message::ShortDebugString, Message::Utf8DebugString) in v30 to redact sensitive fields annotated by debug_redact; the outputs of these APIs will contain a per-process randomized prefix, and so will no longer be parseable by Protobuf TextFormat Parsers.

Motivation

Currently Protobuf debug APIs print every field in a proto into human-readable formats. This may lead to privacy incidents where developers accidentally log Protobuf debug outputs containing sensitive fields.

How to Annotate Sensitive Fields

There are two ways to mark fields sensitive:

• Mark a field with the field option debug_redact = true, directly.

  message Foo {
    optional string secret = 1 [debug_redact = true];
  }
  

• If you have already defined a field annotation of type Enum by extending proto2.FieldOptions, and certain values of this annotation are used to annotate fields you would like to redact, then you can annotate these values with debug_redact = true. All the fields that have been annotated with such values will be redacted.

  package my.package;
  
  extend proto2.FieldOptions {
    # The existing field annotation
    optional ContentType content_type = 1234567;
  };
  
  enum ContentType {
    PUBLIC = 0;
    SECRET = 1 [debug_redact = true];
  };
  
  message Foo {
    # will not be redacted
    optional string public_info = 1 [
      (my.package.content_type) = PUBLIC
    ];
    # will be redacted
    optional string secret = 1 [
      (my.package.content_type) = SECRET
    ];
  }
  

New Debug Format

Compared to the existing debug format, the new debug format has two major differences:

• The sensitive fields annotated with debug_redact are redacted automatically in the output formats
• The output formats will contain a per-process randomized prefix, which will make them no longer be parsable by TextFormat parsers.

Note that the second change is true regardless of whether the proto contains sensitive fields or not, which ensures that any debug output always cannot be deserialized regardless of the proto content.