proto.md

Protocol Documentation

Deckard Service

Method Name	Request Type	Response Type	Description
Add	AddRequest	AddResponse	Adds messages to the queue. If any message already exists (same id and queue) it will be updated
Pull	PullRequest	PullResponse	Pulls messages from the queue
Ack	AckRequest	AckResponse	Acks a message, marking it as successfully processed
Nack	AckRequest	AckResponse	Nacks a message, marking it as not processed
Count	CountRequest	CountResponse	Counts number of messages on queue based on a filter request
Remove	RemoveRequest	RemoveResponse	Removes one or many messages from the queue based on its ids
Flush	FlushRequest	FlushResponse	Discards all deckard data from storage and cache. This request is only available if deckard instance is configured with MEMORY cache and storage.
GetById	GetByIdRequest	GetByIdResponse	Gets a message from a specific queue using its id CAUTION: this should be used mainly for diagnostics and debugging purposes since it will be direct operation on the storage system
EditQueue	EditQueueRequest	EditQueueResponse	Edits a queue configuration
GetQueue	GetQueueRequest	GetQueueResponse	Gets the current configuration for a queue

Messages

AckRequest

Field	Type	Label	Description
id	string		ID of the message
queue	string		Queue where this message is stored
reason	string		Reason of this result. Useful for audit, mostly on 'nack' signals.
score_subtract	double		Deprecated. This field is deprecated and will be removed in the future. If you need to change the message score, use the 'score' field. The value to subtract the score and increase final message priority. For example if you want to make this message to have a higher priority you can set 10000 which will represent 10s of score benefit in the default score algorithm. If you want to penalize the message you can send a negative number. IMPORTANT: The message will not be locked by, in the example, 10 seconds. This field is used only to increase or decrease the message priority in the priority queue. This field is used only for ack requests (since in nack requests the message will return with the lowest score to the queue). It will be ignored if used at the same time of 'score' or 'lock_ms' fields.
breakpoint	string		Breakpoint is a field to be used as an auxiliar field for some specific use cases. For example if you need to keep a record of the last result processing a message, or want to iteract with a pagination system. Examples: imagine a message representing a web news portal and you want to navigate through the articles. This field could be used to store the last visited article id. Or imagine a message representing a user and you want to iterate through the user's publications pages. This field could be used to store the last page number you visited.
lock_ms	int64		Time in milliseconds to lock a message before returning it to the queue. For NACK requests the message will be locked before returning to first position in the priority queue. You can change this behavior using the 'score' field. For ACK requests the message will be locked before returning to last position in the priority queue. You can change this behavior using the 'score' field. IMPORTANT: Deckard checks for locked messages in a 1-second precision meaning the lock have a second precision and not milliseconds. This field is in milliseconds because all duration units on deckard are expressed in milliseconds and the default score algorithm uses milliseconds as well.
removeMessage	bool		Whether the message should be removed when acked/nacked
score	double		Sets the score of the message when ACKed, to override the default score algorithm. If used at the same time with the 'lock_ms' attribute, the message will be locked for the specified time and then returned to the queue with the specified score. For ACK requests, if the score is not provided (or set to 0), the message will return to the queue with the default score algorithm which is the current timestamp in milliseconds. For NACKs requests, if the score is not provided (or set to 0), the message will return to the queue with the minimum score accepted by Deckard which is 0. Negative values will be converted to 0, which is how to set the highest priority to a message in a ACK/NACK request. REMEMBER: the maximum score accepted by Deckard is 9007199254740992 and the minimum is 0, so values outside this range will be capped.

AckResponse

Field	Type	Label	Description
success	bool
removal_response	RemoveResponse		The removal response if the message's removal was requested

AddMessage

Field	Type	Label	Description
id	string		Unique id of this message
payload	AddMessage.PayloadEntry	repeated	A payload map with formatted data to be stored and used by clients.
string_payload	string		Non-formatted string payload. This field can be used to store simple string data instead of using the payload field.
metadata	AddMessage.MetadataEntry	repeated	Metadata is a map of string to be used as a key-value store. It is a simple way to store data that is not part of the message payload.
queue	string		Name of the queue to add this message Suggestion: to better observability, provide the name of the application using colon as the separator. Example: <application_name>:<queue_name> You may also add a queue prefix to the queue name using two colons as the separator. Example: <application_name>:<queue_prefix>::<queue_name>
timeless	bool		Indicate this message will never expire and will only be deleted from the queue if explicitly removed.
ttl_minutes	int64		TTL is the amount of time in minutes this message will live in the queue. TTL is not a exact time, the message may live a little longer than the specified TTL.
description	string		Description of the message, this should be used as a human readable string to be used in diagnostics.
score	double		Score represents the priority score the message currently have in the queue. The score is used to determine the order of the messages returned in a pull request. The lower the score, the higher the priority. If the score is not set (or set to 0), the score will be set with the current timestamp in milliseconds at the moment of the message creation. The maximum score accepted by Deckard is 9007199254740992 and the minimum is 0 Negative scores will be converted to 0, adding the message with the lowest score (and highest priority)

AddMessage.MetadataEntry

Field	Type	Label	Description
key	string
value	string

AddMessage.PayloadEntry

Field	Type	Label	Description
key	string
value	google.protobuf.Any

AddRequest

Field	Type	Label	Description
messages	AddMessage	repeated	List of messages to be added

AddResponse

Field	Type	Label	Description
created_count	int64		Number of created messages
updated_count	int64		Number of updated messages

CountRequest

Field	Type	Label	Description
queue	string

CountResponse

Field	Type	Label	Description
count	int64

EditQueueRequest

Field	Type	Label	Description
queue	string		Name of the queue to be updated This includes all prefixes and suffixes
configuration	QueueConfiguration		Configuration to apply to the queue. It will always update the queue with the newer configuration. Only available fields will be updated, meaning that previously configured fields will not be change unless you explicit set it. If you want to change a configuration to its default value, manually set it to its default value following each field documentation.

EditQueueResponse

Field	Type	Label	Description
queue	string		Name of the queue
success	bool

FlushRequest

FlushResponse

Field	Type	Label	Description
success	bool

GetByIdRequest

Field	Type	Label	Description
queue	string		Name of the queue to get a message
id	string		The message id

GetByIdResponse

Field	Type	Label	Description
message	Message
human_readable_payload	GetByIdResponse.HumanReadablePayloadEntry	repeated	A human readable string data map of the message's payload. This represents the payload map as a JSON string representation, useful for debugging and diagnostics
found	bool

GetByIdResponse.HumanReadablePayloadEntry

Field	Type	Label	Description
key	string
value	string

GetQueueRequest

Field	Type	Label	Description
queue	string		Name of the queue to be updated This includes all prefixes and suffixes

GetQueueResponse

Field	Type	Label	Description
queue	string		Name of the queue
configuration	QueueConfiguration		Configuration of the queue

Message

Field	Type	Label	Description
id	string		ID is an unique identifier inside a queue. Any message with the same id and queue will be considered the same message.
description	string		Description of the message, this should be used as a human readable string to be used in diagnostics.
queue	string		Full name of the queue this message belongs (including any prefixes)
payload	Message.PayloadEntry	repeated	A payload map with formatted data to be stored and used by clients.
metadata	Message.MetadataEntry	repeated	Metadata is a map of string to be used as a key-value store. It is a simple way to store data that is not part of the message payload.
string_payload	string		Message string payload. Is responsibility of the caller to know how to encode/decode to a useful format for its purpose. This field can be used to store simple string data instead of using the payload field.
score	double		Score represents the priority score the message currently have in the queue. The lower the score, the higher the priority. The maximum score accepted by Deckard is 9007199254740992 and the minimum is 0
breakpoint	string		Breakpoint is a field to be used as an auxiliar field for some specific use cases. For example if you need to keep a record of the last result processing a message, or want to iteract with a pagination system. Examples: imagine a message representing a web news portal and you want to navigate through the articles. This field could be used to store the last visited article id. Or imagine a message representing a user and you want to iterate through the user's publications pages. This field could be used to store the last page number you visited.
diagnostics	MessageDiagnostics		Diagnostics is a field holding information about the message's usage. It is useful to track how many times a message was ACKed or NACKed.

Message.MetadataEntry

Field	Type	Label	Description
key	string
value	string

Message.PayloadEntry

Field	Type	Label	Description
key	string
value	google.protobuf.Any

MessageDiagnostics

Field	Type	Label	Description
acks	int64		Track total number of ACKs
nacks	int64		Track total number of NACKs
consecutive_acks	int64		Track number of consecutive ACKs This field will reset to 0 when a NACK is received
consecutive_nacks	int64		Track number of consecutive NACKs This field will reset to 0 when an ACK is received

PullRequest

Field	Type	Label	Description
queue	string		Full name of the queue to pull messages (including any prefixes)
amount	int32		Number of messages to fetch. Caution: as greater the amount, as more time it will take to process the request. Max value is 1000 and the default value is 1
score_filter	int64		Deprecated. Prefer using the max_score field instead of this one. This field is deprecated and will be removed in the future. The score_filter behaves differently than max_score field. The max_score field is the upper threshold itself, but the score_filter will result in an upper score threshold of the current timestamp minus the score_filter value. Useful only when your queue's score is only based on the current timestamp to not return a message just moments after it was last used. It will only return messages with score lower than the current timestamp minus the score_filter value. For example if your queue's score is only based on the current timestamp, this parameter will be the number of milliseconds a message must be in the queue before being returned.
max_score	double		Sets the upper threshold for the priority score of a message to be returned in the pull request. Only messages with a priority score equal to or lower than the max_score value will be returned. The maximum score accepted by Deckard is 9007199254740992, any value higher than this will be capped to the maximum score. To set this value to the minimum score accepted by Deckard, use any negative number. This parameter will be ignored if set to 0 (default value).
min_score	double		Sets the lower threshold for the priority score required for a message to be returned. Only messages with a priority score equal to or higher than the min_score value will be returned. The minimum score accepted by Deckard is 0 which is also the default value
ack_deadline_ms	int64		Sets the amount of time in milliseconds a message will wait be available for processing before being returned to the queue if not ACKed. The default value is 300000 (5 minutes).

PullResponse

Field	Type	Label	Description
messages	Message	repeated	List of returned messages

QueueConfiguration

The queue configuration does not change instantly and can take up to 10 minutes to complete update.

Field	Type	Label	Description
max_elements	int64		Number of max elements the queue can have. To apply a max elements to a queue, set a value greater than 0. To remove the max elements from a queue, set the value to -1. 0 will be always ignored and the queue will not be updated. All queues are unlimited by default. The exclusion policy will be applied to the queue when the max elements is reached: Messages are excluded ordered by its TTL, where the closest to expire will be excluded first. If all messages have the same TTL, the oldest message will be excluded first.

RemoveRequest

Field	Type	Label	Description
ids	string	repeated
queue	string		Name of the queue to remove message Provide the name of the application as a prefix using colon as the separator. Example: <application_name>:<queue_name>

RemoveResponse

Field	Type	Label	Description
cacheRemoved	int64
storageRemoved	int64

Scalar Value Types

.proto Type	Notes	C++	Java	Python	Go	C#	PHP	Ruby
double		double	double	float	float64	double	float	Float
float		float	float	float	float32	float	float	Float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long	int64	long	integer/string	Bignum
uint32	Uses variable-length encoding.	uint32	int	int/long	uint32	uint	integer	Bignum or Fixnum (as required)
uint64	Uses variable-length encoding.	uint64	long	int/long	uint64	ulong	integer/string	Bignum or Fixnum (as required)
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long	int64	long	integer/string	Bignum
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int	uint32	uint	integer	Bignum or Fixnum (as required)
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long	uint64	ulong	integer/string	Bignum
sfixed32	Always four bytes.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sfixed64	Always eight bytes.	int64	long	int/long	int64	long	integer/string	Bignum
bool		bool	boolean	boolean	bool	bool	boolean	TrueClass/FalseClass
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode	string	string	string	String (UTF-8)
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str	[]byte	ByteString	string	String (ASCII-8BIT)

Google Protobuf Any

The Any type is a special type that can represent any other type.

The Any message type lets you use messages as embedded types without having their .proto definition. An Any contains an arbitrary serialized message as bytes, along with a URL that acts as a globally unique identifier for and resolves to that message's type.

Documentation on how to use this field for each language:

• C++
• Java
• Python
• Go
• C#
• PHP
• Ruby