coded_stream.h

This section contains reference documentation for working with protocol buffer classes in C++.

#include <google/protobuf/io/coded_stream.h>
namespace google::protobuf::io

This file contains the CodedInputStream and CodedOutputStream classes, which wrap a ZeroCopyInputStream or ZeroCopyOutputStream, respectively, and allow you to read or write individual pieces of data in various formats.

In particular, these implement the varint encoding for integers, a simple variable-length encoding in which smaller numbers take fewer bytes.

Typically these classes will only be used internally by the protocol buffer library in order to encode and decode protocol buffers. Clients of the library only need to know about this class if they wish to write custom message parsing or serialization procedures.

CodedOutputStream example:

// Write some data to "myfile".  First we write a 4-byte "magic number"
// to identify the file type, then write a length-delimited string.  The
// string is composed of a varint giving the length followed by the raw
// bytes.
int fd = open("myfile", O_CREAT | O_WRONLY);
ZeroCopyOutputStream* raw_output = new FileOutputStream(fd);
CodedOutputStream* coded_output = new CodedOutputStream(raw_output);

int magic_number = 1234;
char text[] = "Hello world!";
coded_output->WriteLittleEndian32(magic_number);
coded_output->WriteVarint32(strlen(text));
coded_output->WriteRaw(text, strlen(text));

delete coded_output;
delete raw_output;
close(fd);

CodedInputStream example:

// Read a file created by the above code.
int fd = open("myfile", O_RDONLY);
ZeroCopyInputStream* raw_input = new FileInputStream(fd);
CodedInputStream* coded_input = new CodedInputStream(raw_input);

coded_input->ReadLittleEndian32(&magic_number);
if (magic_number != 1234) {
  cerr << "File not in expected format." << endl;
  return;
}

uint32 size;
coded_input->ReadVarint32(&size);

char* text = new char[size + 1];
coded_input->ReadRaw(buffer, size);
text[size] = '\0';

delete coded_input;
delete raw_input;
close(fd);

cout << "Text is: " << text << endl;
delete [] text;

For those who are interested, varint encoding is defined as follows:

The encoding operates on unsigned integers of up to 64 bits in length. Each byte of the encoded value has the format:

  • bits 0-6: Seven bits of the number being encoded.
  • bit 7: Zero if this is the last byte in the encoding (in which case all remaining bits of the number are zero) or 1 if more bytes follow. The first byte contains the least-significant 7 bits of the number, the second byte (if present) contains the next-least-significant 7 bits, and so on. So, the binary number 1011000101011 would be encoded in two bytes as "10101011 00101100".

In theory, varint could be used to encode integers of any length. However, for practicality we set a limit at 64 bits. The maximum encoded length of a number is thus 10 bytes.

Classes in this file

Class which reads and decodes binary data which is composed of varint- encoded integers and fixed-width pieces.
EpsCopyOutputStream wraps a ZeroCopyOutputStream and exposes a new stream, which has the property you can write kSlopBytes (16 bytes) from the current position without bounds checks.
Class which encodes and writes binary data which is composed of varint- encoded integers and fixed-width pieces.
Compile-time equivalent of VarintSize32().

class CodedInputStream

#include <google/protobuf/io/coded_stream.h>
namespace google::protobuf::io

Class which reads and decodes binary data which is composed of varint- encoded integers and fixed-width pieces.

Wraps a ZeroCopyInputStream. Most users will not need to deal with CodedInputStream.

Most methods of CodedInputStream that return a bool return false if an underlying I/O error occurs or if the data is malformed. Once such a failure occurs, the CodedInputStream is broken and is no longer useful. After a failure, callers also should assume writes to "out" args may have occurred, though nothing useful can be determined from those writes.

Members

explicit
CodedInputStream(ZeroCopyInputStream * input)
Create a CodedInputStream that reads from the given ZeroCopyInputStream.
explicit
CodedInputStream(const uint8 * buffer, int size)
Create a CodedInputStream that reads from the given flat array. more...
~CodedInputStream()
Destroy the CodedInputStream and position the underlying ZeroCopyInputStream at the first unread byte. more...
bool
IsFlat() const
Return true if this CodedInputStream reads from a flat array instead of a ZeroCopyInputStream.
bool
Skip(int count)
Skips a number of bytes. more...
bool
GetDirectBufferPointer(const void ** data, int * size)
Sets *data to point directly at the unread part of the CodedInputStream's underlying buffer, and *size to the size of that buffer, but does not advance the stream's current position. more...
void
GetDirectBufferPointerInline(const void ** data, int * size)
Like GetDirectBufferPointer, but this method is inlined, and does not attempt to Refresh() if the buffer is currently empty.
bool
ReadRaw(void * buffer, int size)
Read raw bytes, copying them into the given buffer.
bool
ReadString(std::string * buffer, int size)
Like ReadRaw, but reads into a string.
bool
ReadLittleEndian32(uint32 * value)
Read a 32-bit little-endian integer.
bool
ReadLittleEndian64(uint64 * value)
Read a 64-bit little-endian integer.
bool
ReadVarint32(uint32 * value)
Read an unsigned integer with Varint encoding, truncating to 32 bits. more...
bool
ReadVarint64(uint64 * value)
Read an unsigned integer with Varint encoding.
bool
ReadVarintSizeAsInt(int * value)
Reads a varint off the wire into an "int". more...
uint32
ReadTag()
Read a tag. more...
uint32
ReadTagNoLastTag()
std::pair< uint32, bool >
ReadTagWithCutoff(uint32 cutoff)
This usually a faster alternative to ReadTag() when cutoff is a manifest constant. more...
std::pair< uint32, bool >
ReadTagWithCutoffNoLastTag(uint32 cutoff)
bool
ExpectTag(uint32 expected)
Usually returns true if calling ReadVarint32() now would produce the given value. more...
bool
ExpectAtEnd()
Usually returns true if no more bytes can be read. more...
bool
LastTagWas(uint32 expected)
If the last call to ReadTag() or ReadTagWithCutoff() returned the given value, returns true. more...
void
SetLastTag(uint32 tag)
bool
ConsumedEntireMessage()
When parsing message (but NOT a group), this method must be called immediately after MergeFromCodedStream() returns (if it returns true) to further verify that the message ended in a legitimate way. more...
void
SetConsumed()
static const uint8 *
ReadLittleEndian32FromArray(const uint8 * buffer, uint32 * value)
These methods read from an externally provided buffer. more...
static const uint8 *
ReadLittleEndian64FromArray(const uint8 * buffer, uint64 * value)
Read a 64-bit little-endian integer. more...
static const uint8 *
ExpectTagFromArray(const uint8 * buffer, uint32 expected)
Like above, except this reads from the specified buffer. more...

Limits

Limits are used when parsing length-delimited embedded messages. After the message's length is read, PushLimit() is used to prevent the CodedInputStream from reading beyond that length. Once the embedded message has been parsed, PopLimit() is called to undo the limit.
typedef
int Limit
Opaque type used with PushLimit() and PopLimit(). more...
Limit
PushLimit(int byte_limit)
Places a limit on the number of bytes that the stream may read, starting from the current position. more...
void
PopLimit(Limit limit)
Pops the last limit pushed by PushLimit(). more...
int
BytesUntilLimit() const
Returns the number of bytes left until the nearest limit on the stack is hit, or -1 if no limits are in place.
int
CurrentPosition() const
Returns current position relative to the beginning of the input stream.

Total Bytes Limit

To prevent malicious users from sending excessively large messages and causing memory exhaustion, CodedInputStream imposes a hard limit on the total number of bytes it will read.
int = { SetTotalBytesLimit(total_bytes_limit)
void
SetTotalBytesLimit(int total_bytes_limit)
Sets the maximum number of bytes that this CodedInputStream will read before refusing to continue. more...
PROTOBUF_DEPRECATED_MSG("Please use the single parameter version of SetTotalBytesLimit(). The " "second parameter is ignored." )
int
BytesUntilTotalBytesLimit() const
The Total Bytes Limit minus the Current Position, or -1 if the total bytes limit is INT_MAX.

Recursion Limit

To prevent corrupt or malicious messages from causing stack overflows, we must keep track of the depth of recursion when parsing embedded messages and groups.

CodedInputStream keeps track of this because it is the only object that is passed down the stack during parsing.

void
SetRecursionLimit(int limit)
Sets the maximum recursion depth. The default is 100.
int
RecursionBudget()
bool
IncrementRecursionDepth()
Increments the current recursion depth. more...
void
DecrementRecursionDepth()
Decrements the recursion depth if possible.
void
UnsafeDecrementRecursionDepth()
Decrements the recursion depth blindly. more...
std::pair< CodedInputStream::Limit, int >
IncrementRecursionDepthAndPushLimit(int byte_limit)
Shorthand for make_pair(PushLimit(byte_limit), –recursion_budget_). more...
Limit
ReadLengthAndPushLimit()
Shorthand for PushLimit(ReadVarint32(&length) ? length : 0).
bool
DecrementRecursionDepthAndPopLimit(Limit limit)
Helper that is equivalent to: { bool result = ConsumedEntireMessage(); PopLimit(limit); UnsafeDecrementRecursionDepth(); return result; } Using this can reduce code size and complexity in some cases. more...
bool
CheckEntireMessageConsumedAndPopLimit(Limit limit)
Helper that is equivalent to: { bool result = ConsumedEntireMessage(); PopLimit(limit); return result; } Using this can reduce code size and complexity in some cases.
static int
GetDefaultRecursionLimit()

Extension Registry

ADVANCED USAGE: 99.9% of people can ignore this section.

By default, when parsing extensions, the parser looks for extension definitions in the pool which owns the outer message's Descriptor. However, you may call SetExtensionRegistry() to provide an alternative pool instead. This makes it possible, for example, to parse a message using a generated class, but represent some extensions using DynamicMessage.

void
SetExtensionRegistry(const DescriptorPool * pool, MessageFactory * factory)
Set the pool used to look up extensions. more...
const DescriptorPool *
GetExtensionPool()
Get the DescriptorPool set via SetExtensionRegistry(), or NULL if no pool has been provided.
MessageFactory *
GetExtensionFactory()
Get the MessageFactory set via SetExtensionRegistry(), or NULL if no factory has been provided.

explicit CodedInputStream::CodedInputStream(
        const uint8 * buffer,
        int size)

Create a CodedInputStream that reads from the given flat array.

This is faster than using an ArrayInputStream. PushLimit(size) is implied by this constructor.


CodedInputStream::~CodedInputStream()

Destroy the CodedInputStream and position the underlying ZeroCopyInputStream at the first unread byte.

If an error occurred while reading (causing a method to return false), then the exact position of the input stream may be anywhere between the last value that was read successfully and the stream's byte limit.


bool CodedInputStream::Skip(
        int count)

Skips a number of bytes.

Returns false if an underlying read error occurs.


bool CodedInputStream::GetDirectBufferPointer(
        const void ** data,
        int * size)

Sets *data to point directly at the unread part of the CodedInputStream's underlying buffer, and *size to the size of that buffer, but does not advance the stream's current position.

This will always either produce a non-empty buffer or return false. If the caller consumes any of this data, it should then call Skip() to skip over the consumed bytes. This may be useful for implementing external fast parsing routines for types of data not covered by the CodedInputStream interface.


bool CodedInputStream::ReadVarint32(
        uint32 * value)

Read an unsigned integer with Varint encoding, truncating to 32 bits.

Reading a 32-bit value is equivalent to reading a 64-bit one and casting it to uint32, but may be more efficient.


bool CodedInputStream::ReadVarintSizeAsInt(
        int * value)

Reads a varint off the wire into an "int".

This should be used for reading sizes off the wire (sizes of strings, submessages, bytes fields, etc).

The value from the wire is interpreted as unsigned. If its value exceeds the representable value of an integer on this platform, instead of truncating we return false. Truncating (as performed by ReadVarint32() above) is an acceptable approach for fields representing an integer, but when we are parsing a size from the wire, truncating the value would result in us misparsing the payload.


uint32 CodedInputStream::ReadTag()

Read a tag.

This calls ReadVarint32() and returns the result, or returns zero (which is not a valid tag) if ReadVarint32() fails. Also, ReadTag (but not ReadTagNoLastTag) updates the last tag value, which can be checked with LastTagWas().

Always inline because this is only called in one place per parse loop but it is called for every iteration of said loop, so it should be fast. GCC doesn't want to inline this by default.


std::pair< uint32, bool >
    CodedInputStream::ReadTagWithCutoff(
        uint32 cutoff)

This usually a faster alternative to ReadTag() when cutoff is a manifest constant.

It does particularly well for cutoff >= 127. The first part of the return value is the tag that was read, though it can also be 0 in the cases where ReadTag() would return 0. If the second part is true then the tag is known to be in [0, cutoff]. If not, the tag either is above cutoff or is 0. (There's intentional wiggle room when tag is 0, because that can arise in several ways, and for best performance we want to avoid an extra "is tag == 0?" check here.)


bool CodedInputStream::ExpectTag(
        uint32 expected)

Usually returns true if calling ReadVarint32() now would produce the given value.

Will always return false if ReadVarint32() would not return the given value. If ExpectTag() returns true, it also advances past the varint. For best performance, use a compile-time constant as the parameter. Always inline because this collapses to a small number of instructions when given a constant parameter, but GCC doesn't want to inline by default.


bool CodedInputStream::ExpectAtEnd()

Usually returns true if no more bytes can be read.

Always returns false if more bytes can be read. If ExpectAtEnd() returns true, a subsequent call to LastTagWas() will act as if ReadTag() had been called and returned zero, and ConsumedEntireMessage() will return true.


bool CodedInputStream::LastTagWas(
        uint32 expected)

If the last call to ReadTag() or ReadTagWithCutoff() returned the given value, returns true.

Otherwise, returns false. ReadTagNoLastTag/ReadTagWithCutoffNoLastTag do not preserve the last returned value.

This is needed because parsers for some types of embedded messages (with field type TYPE_GROUP) don't actually know that they've reached the end of a message until they see an ENDGROUP tag, which was actually part of the enclosing message. The enclosing message would like to check that tag to make sure it had the right number, so it calls LastTagWas() on return from the embedded parser to check.


bool CodedInputStream::ConsumedEntireMessage()

When parsing message (but NOT a group), this method must be called immediately after MergeFromCodedStream() returns (if it returns true) to further verify that the message ended in a legitimate way.

For example, this verifies that parsing did not end on an end-group tag. It also checks for some cases where, due to optimizations, MergeFromCodedStream() can incorrectly return true.


static const uint8 * CodedInputStream::ReadLittleEndian32FromArray(
        const uint8 * buffer,
        uint32 * value)

These methods read from an externally provided buffer.

static

The caller is responsible for ensuring that the buffer has sufficient space. Read a 32-bit little-endian integer.


static const uint8 * CodedInputStream::ReadLittleEndian64FromArray(
        const uint8 * buffer,
        uint64 * value)

Read a 64-bit little-endian integer.

static


static const uint8 * CodedInputStream::ExpectTagFromArray(
        const uint8 * buffer,
        uint32 expected)

Like above, except this reads from the specified buffer.

The caller is responsible for ensuring that the buffer is large enough to read a varint of the expected size. For best performance, use a compile-time constant as the expected tag parameter.

Returns a pointer beyond the expected tag if it was found, or NULL if it was not.


typedef CodedInputStream::Limit

Opaque type used with PushLimit() and PopLimit().

Do not modify values of this type yourself. The only reason that this isn't a struct with private internals is for efficiency.


Limit CodedInputStream::PushLimit(
        int byte_limit)

Places a limit on the number of bytes that the stream may read, starting from the current position.

Once the stream hits this limit, it will act like the end of the input has been reached until PopLimit() is called.

As the names imply, the stream conceptually has a stack of limits. The shortest limit on the stack is always enforced, even if it is not the top limit.

The value returned by PushLimit() is opaque to the caller, and must be passed unchanged to the corresponding call to PopLimit().


void CodedInputStream::PopLimit(
        Limit limit)

Pops the last limit pushed by PushLimit().

The input must be the value returned by that call to PushLimit().


void CodedInputStream::SetTotalBytesLimit(
        int total_bytes_limit)

Sets the maximum number of bytes that this CodedInputStream will read before refusing to continue.

To prevent servers from allocating enormous amounts of memory to hold parsed messages, the maximum message length should be limited to the shortest length that will not harm usability. The default limit is INT_MAX (~2GB) and apps should set shorter limits if possible. An error will always be printed to stderr if the limit is reached.

Note: setting a limit less than the current read position is interpreted as a limit on the current position.

This is unrelated to PushLimit()/PopLimit().


bool CodedInputStream::IncrementRecursionDepth()

Increments the current recursion depth.

Returns true if the depth is under the limit, false if it has gone over.


void CodedInputStream::UnsafeDecrementRecursionDepth()

Decrements the recursion depth blindly.

This is faster than DecrementRecursionDepth(). It should be used only if all previous increments to recursion depth were successful.


std::pair< CodedInputStream::Limit, int >
    CodedInputStream::IncrementRecursionDepthAndPushLimit(
        int byte_limit)

Shorthand for make_pair(PushLimit(byte_limit), –recursion_budget_).

Using this can reduce code size and complexity in some cases. The caller is expected to check that the second part of the result is non-negative (to bail out if the depth of recursion is too high) and, if all is well, to later pass the first part of the result to PopLimit() or similar.


bool CodedInputStream::DecrementRecursionDepthAndPopLimit(
        Limit limit)

Helper that is equivalent to: { bool result = ConsumedEntireMessage(); PopLimit(limit); UnsafeDecrementRecursionDepth(); return result; } Using this can reduce code size and complexity in some cases.

Do not use unless the current recursion depth is greater than zero.


void CodedInputStream::SetExtensionRegistry(
        const DescriptorPool * pool,
        MessageFactory * factory)

Set the pool used to look up extensions.

Most users do not need to call this as the correct pool will be chosen automatically.

WARNING: It is very easy to misuse this. Carefully read the requirements below. Do not use this unless you are sure you need it. Almost no one does.

Let's say you are parsing a message into message object m, and you want to take advantage of SetExtensionRegistry(). You must follow these requirements:

The given DescriptorPool must contain m->GetDescriptor(). It is not sufficient for it to simply contain a descriptor that has the same name and content – it must be the exact object. In other words:

assert(pool->FindMessageTypeByName(m->GetDescriptor()->full_name()) ==
       m->GetDescriptor());

There are two ways to satisfy this requirement: 1) Use m->GetDescriptor()->pool() as the pool. This is generally useless

because this is the pool that would be used anyway if you didn't call
SetExtensionRegistry() at all.

2) Use a DescriptorPool which has m->GetDescriptor()->pool() as an

"underlay".  Read the documentation for DescriptorPool for more
information about underlays.

You must also provide a MessageFactory. This factory will be used to construct Message objects representing extensions. The factory's GetPrototype() MUST return non-NULL for any Descriptor which can be found through the provided pool.

If the provided factory might return instances of protocol-compiler- generated (i.e. compiled-in) types, or if the outer message object m is a generated type, then the given factory MUST have this property: If GetPrototype() is given a Descriptor which resides in DescriptorPool::generated_pool(), the factory MUST return the same prototype which MessageFactory::generated_factory() would return. That is, given a descriptor for a generated type, the factory must return an instance of the generated class (NOT DynamicMessage). However, when given a descriptor for a type that is NOT in generated_pool, the factory is free to return any implementation.

The reason for this requirement is that generated sub-objects may be accessed via the standard (non-reflection) extension accessor methods, and these methods will down-cast the object to the generated class type. If the object is not actually of that type, the results would be undefined. On the other hand, if an extension is not compiled in, then there is no way the code could end up accessing it via the standard accessors – the only way to access the extension is via reflection. When using reflection, DynamicMessage and generated messages are indistinguishable, so it's fine if these objects are represented using DynamicMessage.

Using DynamicMessageFactory on which you have called SetDelegateToGeneratedFactory(true) should be sufficient to satisfy the above requirement.

If either pool or factory is NULL, both must be NULL.

Note that this feature is ignored when parsing "lite" messages as they do not have descriptors.

class EpsCopyOutputStream

#include <google/protobuf/io/coded_stream.h>
namespace google::protobuf::io

EpsCopyOutputStream wraps a ZeroCopyOutputStream and exposes a new stream, which has the property you can write kSlopBytes (16 bytes) from the current position without bounds checks.

The cursor into the stream is managed by the user of the class and is an explicit parameter in the methods. Careful use of this class, ie. keep ptr a local variable, eliminates the need to for the compiler to sync the ptr value between register and memory.

Members

enum
@33
EpsCopyOutputStream(ZeroCopyOutputStream * stream, bool deterministic, uint8 ** pp)
Initialize from a stream.
EpsCopyOutputStream(void * data, int size, bool deterministic)
Only for array serialization. more...
EpsCopyOutputStream(void * data, int size, ZeroCopyOutputStream * stream, bool deterministic, uint8 ** pp)
Initialize from stream but with the first buffer already given (eager).
uint8 *
Trim(uint8 * ptr)
Flush everything that's written into the underlying ZeroCopyOutputStream and trims the underlying stream to the location of ptr.
PROTOBUF_MUST_USE_RESULT uint8 *
EnsureSpace(uint8 * ptr)
After this it's guaranteed you can safely write kSlopBytes to ptr. more...
uint8 *
WriteRaw(const void * data, int size, uint8 * ptr)
uint8 *
WriteRawMaybeAliased(const void * data, int size, uint8 * ptr)
Writes the buffer specified by data, size to the stream. more...
uint8 *
WriteStringMaybeAliased(uint32 num, const std::string & s, uint8 * ptr)
uint8 *
WriteBytesMaybeAliased(uint32 num, const std::string & s, uint8 * ptr)
template uint8 *
WriteString(uint32 num, const T & s, uint8 * ptr)
template uint8 *
WriteBytes(uint32 num, const T & s, uint8 * ptr)
template uint8 *
WriteInt32Packed(int num, const T & r, int size, uint8 * ptr)
template uint8 *
WriteUInt32Packed(int num, const T & r, int size, uint8 * ptr)
template uint8 *
WriteSInt32Packed(int num, const T & r, int size, uint8 * ptr)
template uint8 *
WriteInt64Packed(int num, const T & r, int size, uint8 * ptr)
template uint8 *
WriteUInt64Packed(int num, const T & r, int size, uint8 * ptr)
template uint8 *
WriteSInt64Packed(int num, const T & r, int size, uint8 * ptr)
template uint8 *
WriteEnumPacked(int num, const T & r, int size, uint8 * ptr)
template uint8 *
WriteFixedPacked(int num, const T & r, uint8 * ptr)
bool
HadError() const
Returns true if there was an underlying I/O error since this object was created.
void
EnableAliasing(bool enabled)
Instructs the EpsCopyOutputStream to allow the underlying ZeroCopyOutputStream to hold pointers to the original structure instead of copying, if it supports it (i.e. more...
void
SetSerializationDeterministic(bool value)
bool
IsSerializationDeterministic() const
int64
ByteCount(uint8 * ptr) const
The number of bytes written to the stream at position ptr, relative to the stream's overall position.
uint8 *
SetInitialBuffer(void * data, int size)
These methods are for CodedOutputStream. more...

enum EpsCopyOutputStream::@33 {
  kSlopBytes = = 16
}

kSlopBytes

EpsCopyOutputStream::EpsCopyOutputStream(
        void * data,
        int size,
        bool deterministic)

Only for array serialization.

No overflow protection, end_ will be the pointed to the end of the array. When using this the total size is already known, so no need to maintain the slop region.


PROTOBUF_MUST_USE_RESULT uint8 *
    EpsCopyOutputStream::EnsureSpace(
        uint8 * ptr)

After this it's guaranteed you can safely write kSlopBytes to ptr.

This will never fail! The underlying stream can produce an error. Use HadError to check for errors.


uint8 * EpsCopyOutputStream::WriteRawMaybeAliased(
        const void * data,
        int size,
        uint8 * ptr)

Writes the buffer specified by data, size to the stream.

Possibly by aliasing the buffer (ie. not copying the data). The caller is responsible to make sure the buffer is alive for the duration of the ZeroCopyOutputStream.


void EpsCopyOutputStream::EnableAliasing(
        bool enabled)

Instructs the EpsCopyOutputStream to allow the underlying ZeroCopyOutputStream to hold pointers to the original structure instead of copying, if it supports it (i.e.

output->AllowsAliasing() is true). If the underlying stream does not support aliasing, then enabling it has no affect. For now, this only affects the behavior of WriteRawMaybeAliased().

NOTE: It is caller's responsibility to ensure that the chunk of memory remains live until all of the data has been consumed from the stream.


uint8 * EpsCopyOutputStream::SetInitialBuffer(
        void * data,
        int size)

These methods are for CodedOutputStream.

Ideally they should be private but to match current behavior of CodedOutputStream as close as possible we allow it some functionality.

class CodedOutputStream

#include <google/protobuf/io/coded_stream.h>
namespace google::protobuf::io

Class which encodes and writes binary data which is composed of varint- encoded integers and fixed-width pieces.

Wraps a ZeroCopyOutputStream. Most users will not need to deal with CodedOutputStream.

Most methods of CodedOutputStream which return a bool return false if an underlying I/O error occurs. Once such a failure occurs, the CodedOutputStream is broken and is no longer useful. The Write* methods do not return the stream status, but will invalidate the stream if an error occurs. The client can probe HadError() to determine the status.

Note that every method of CodedOutputStream which writes some data has a corresponding static "ToArray" version. These versions write directly to the provided buffer, returning a pointer past the last written byte. They require that the buffer has sufficient capacity for the encoded data. This allows an optimization where we check if an output stream has enough space for an entire message before we start writing and, if there is, we call only the ToArray methods to avoid doing bound checks for each individual value. i.e., in the example above:

CodedOutputStream* coded_output = new CodedOutputStream(raw_output);
int magic_number = 1234;
char text[] = "Hello world!";

int coded_size = sizeof(magic_number) + CodedOutputStream::VarintSize32(strlen(text)) + strlen(text);

uint8* buffer = coded_output->GetDirectBufferForNBytesAndAdvance(coded_size); if (buffer != nullptr) { // The output stream has enough space in the buffer: write directly to // the array. buffer = CodedOutputStream::WriteLittleEndian32ToArray(magic_number, buffer); buffer = CodedOutputStream::WriteVarint32ToArray(strlen(text), buffer); buffer = CodedOutputStream::WriteRawToArray(text, strlen(text), buffer); } else { // Make bound-checked writes, which will ask the underlying stream for // more space as needed. coded_output->WriteLittleEndian32(magic_number); coded_output->WriteVarint32(strlen(text)); coded_output->WriteRaw(text, strlen(text)); }

delete coded_output;

Members

explicit
CodedOutputStream(ZeroCopyOutputStream * stream)
Create an CodedOutputStream that writes to the given ZeroCopyOutputStream.
CodedOutputStream(ZeroCopyOutputStream * stream, bool do_eager_refresh)
~CodedOutputStream()
Destroy the CodedOutputStream and position the underlying ZeroCopyOutputStream immediately after the last byte written.
bool
HadError()
Returns true if there was an underlying I/O error since this object was created. more...
void
Trim()
Trims any unused space in the underlying buffer so that its size matches the number of bytes written by this stream. more...
bool
Skip(int count)
Skips a number of bytes, leaving the bytes unmodified in the underlying buffer. more...
bool
GetDirectBufferPointer(void ** data, int * size)
Sets *data to point directly at the unwritten part of the CodedOutputStream's underlying buffer, and *size to the size of that buffer, but does not advance the stream's current position. more...
uint8 *
GetDirectBufferForNBytesAndAdvance(int size)
If there are at least "size" bytes available in the current buffer, returns a pointer directly into the buffer and advances over these bytes. more...
void
WriteRaw(const void * buffer, int size)
Write raw bytes, copying them from the given buffer.
void
WriteRawMaybeAliased(const void * data, int size)
Like WriteRaw() but will try to write aliased data if aliasing is turned on.
void
WriteString(const std::string & str)
Equivalent to WriteRaw(str.data(), str.size()).
void
WriteLittleEndian32(uint32 value)
Write a 32-bit little-endian integer.
void
WriteLittleEndian64(uint64 value)
Write a 64-bit little-endian integer.
void
WriteVarint32(uint32 value)
Write an unsigned integer with Varint encoding. more...
void
WriteVarint64(uint64 value)
Write an unsigned integer with Varint encoding.
void
WriteVarint32SignExtended(int32 value)
Equivalent to WriteVarint32() except when the value is negative, in which case it must be sign-extended to a full 10 bytes.
void
WriteTag(uint32 value)
This is identical to WriteVarint32(), but optimized for writing tags. more...
int
ByteCount() const
Returns the total number of bytes written since this object was created.
void
EnableAliasing(bool enabled)
Instructs the CodedOutputStream to allow the underlying ZeroCopyOutputStream to hold pointers to the original structure instead of copying, if it supports it (i.e. more...
void
SetSerializationDeterministic(bool value)
Indicate to the serializer whether the user wants derministic serialization. more...
bool
IsSerializationDeterministic() const
Return whether the user wants deterministic serialization. See above.
template void
Serialize(const Func & func)
uint8 *
Cur() const
void
SetCur(uint8 * ptr)
EpsCopyOutputStream *
EpsCopy()
static uint8 *
WriteRawToArray(const void * buffer, int size, uint8 * target)
Like WriteRaw() but writing directly to the target array. more...
static uint8 *
WriteStringToArray(const std::string & str, uint8 * target)
Like WriteString() but writing directly to the target array.
static uint8 *
WriteStringWithSizeToArray(const std::string & str, uint8 * target)
Write the varint-encoded size of str followed by str.
static uint8 *
WriteLittleEndian32ToArray(uint32 value, uint8 * target)
Like WriteLittleEndian32() but writing directly to the target array.
static uint8 *
WriteLittleEndian64ToArray(uint64 value, uint8 * target)
Like WriteLittleEndian64() but writing directly to the target array.
static uint8 *
WriteVarint32ToArray(uint32 value, uint8 * target)
Like WriteVarint32() but writing directly to the target array.
static uint8 *
WriteVarint32ToArrayOutOfLine(uint32 value, uint8 * target)
Like WriteVarint32() but writing directly to the target array, and with the less common-case paths being out of line rather than inlined.
static uint8 *
WriteVarint64ToArray(uint64 value, uint8 * target)
Like WriteVarint64() but writing directly to the target array.
static uint8 *
WriteVarint32SignExtendedToArray(int32 value, uint8 * target)
Like WriteVarint32SignExtended() but writing directly to the target array.
static uint8 *
WriteTagToArray(uint32 value, uint8 * target)
Like WriteTag() but writing directly to the target array.
static size_t
VarintSize32(uint32 value)
Returns the number of bytes needed to encode the given value as a varint.
static size_t
VarintSize64(uint64 value)
Returns the number of bytes needed to encode the given value as a varint.
static size_t
VarintSize32SignExtended(int32 value)
If negative, 10 bytes. Otherwise, same as VarintSize32().
static bool
IsDefaultSerializationDeterministic()

bool CodedOutputStream::HadError()

Returns true if there was an underlying I/O error since this object was created.

On should call Trim before this function in order to catch all errors.


void CodedOutputStream::Trim()

Trims any unused space in the underlying buffer so that its size matches the number of bytes written by this stream.

The underlying buffer will automatically be trimmed when this stream is destroyed; this call is only necessary if the underlying buffer is accessed before the stream is destroyed.


bool CodedOutputStream::Skip(
        int count)

Skips a number of bytes, leaving the bytes unmodified in the underlying buffer.

Returns false if an underlying write error occurs. This is mainly useful with GetDirectBufferPointer(). Note of caution, the skipped bytes may contain uninitialized data. The caller must make sure that the skipped bytes are properly initialized, otherwise you might leak bytes from your heap.


bool CodedOutputStream::GetDirectBufferPointer(
        void ** data,
        int * size)

Sets *data to point directly at the unwritten part of the CodedOutputStream's underlying buffer, and *size to the size of that buffer, but does not advance the stream's current position.

This will always either produce a non-empty buffer or return false. If the caller writes any data to this buffer, it should then call Skip() to skip over the consumed bytes. This may be useful for implementing external fast serialization routines for types of data not covered by the CodedOutputStream interface.


uint8 * CodedOutputStream::GetDirectBufferForNBytesAndAdvance(
        int size)

If there are at least "size" bytes available in the current buffer, returns a pointer directly into the buffer and advances over these bytes.

The caller may then write directly into this buffer (e.g. using the *ToArray static methods) rather than go through CodedOutputStream. If there are not enough bytes available, returns NULL. The return pointer is invalidated as soon as any other non-const method of CodedOutputStream is called.


void CodedOutputStream::WriteVarint32(
        uint32 value)

Write an unsigned integer with Varint encoding.

Writing a 32-bit value is equivalent to casting it to uint64 and writing it as a 64-bit value, but may be more efficient.


void CodedOutputStream::WriteTag(
        uint32 value)

This is identical to WriteVarint32(), but optimized for writing tags.

In particular, if the input is a compile-time constant, this method compiles down to a couple instructions. Always inline because otherwise the aforementioned optimization can't work, but GCC by default doesn't want to inline this.


void CodedOutputStream::EnableAliasing(
        bool enabled)

Instructs the CodedOutputStream to allow the underlying ZeroCopyOutputStream to hold pointers to the original structure instead of copying, if it supports it (i.e.

output->AllowsAliasing() is true). If the underlying stream does not support aliasing, then enabling it has no affect. For now, this only affects the behavior of WriteRawMaybeAliased().

NOTE: It is caller's responsibility to ensure that the chunk of memory remains live until all of the data has been consumed from the stream.


void CodedOutputStream::SetSerializationDeterministic(
        bool value)

Indicate to the serializer whether the user wants derministic serialization.

The default when this is not called comes from the global default, controlled by SetDefaultSerializationDeterministic.

What deterministic serialization means is entirely up to the driver of the serialization process (i.e. the caller of methods like WriteVarint32). In the case of serializing a proto buffer message using one of the methods of MessageLite, this means that for a given binary equal messages will always be serialized to the same bytes. This implies:

Repeated serialization of a message will return the same bytes.

Different processes running the same binary (including on different machines) will serialize equal messages to the same bytes.

Note that this is not canonical across languages. It is also unstable across different builds with intervening message definition changes, due to unknown fields. Users who need canonical serialization (e.g. persistent storage in a canonical form, fingerprinting) should define their own canonicalization specification and implement the serializer using reflection APIs rather than relying on this API.


static uint8 * CodedOutputStream::WriteRawToArray(
        const void * buffer,
        int size,
        uint8 * target)

Like WriteRaw() but writing directly to the target array.

This is not inlined, as the compiler often optimizes memcpy into inline copy loops. Since this gets called by every field with string or bytes type, inlining may lead to a significant amount of code bloat, with only a minor performance gain.

template struct CodedOutputStream::StaticVarintSize32

#include <google/protobuf/io/coded_stream.h>
namespace google::protobuf::io

template <typename Value>

Compile-time equivalent of VarintSize32().

Members

const size_t
value = = (Value < (1 << 7)) ? 1 : (Value < (1 << 14)) ? 2 : (Value < (1 << 21)) ? 3 : (Value < (1 << 28)) ? 4 : 5