Copyright © 2025 Staysail Systems, Inc.
San Marcos, California, United States of America

If you find this book or NNG useful, please consider sponsoring the project.

The MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Dedication

— For Jenichka

The sun always shines when you are home.

Preface

Preface for the First Edition

At the time of this writing, we are wrapping up NNG for its formal 1.0.0 release. It’s a good time for reflection on the road that we took to get here. Like the road on the cover of this book, it was windy (if quite a bit longer), but what we find at the end has made the journey worthwhile.

Originally the NNG project was conceived as a relatively modest effort to rewrite nanomsg based on threads, with a more readily extensible internal architecture so that we could more easily undertake projects like the ZeroTier and TLS transports.

It would not be incorrect to say that the initial NNG effort was started in “anger”, as we were frustrated with nanomsg’s very complex internal state machines. Looking back on it now, those complex state state machines don’t seem nearly as insane as they did just a year ago.

The simple, naïve, approach we would have preferred, and the one we originally started with, involved significant use of threads, inspired by the work we did in mangos, which uses Go’s goroutines heavily. Goroutines are excellent. Threads, it turns out, are not. Scalable, asynchronous, portable I/O is a lot harder than it looks.

Our experience with in-kernel threads on illumos and Solaris spoiled us, and left us utterly unprepared for cesspool that really is large amounts of userspace programming.

Instead, we have created our own, completely asynchronous core, giving us advanced multiprocessing and concurrency capabilities, without either sacrificing portability or settling for some unhappy least common denominator. This core is a robust foundation for NNG and handling the “Scalability Protocols”, but if we’re being completely honest, we think this core has braod applicability for beyond just the Scalability Protocols. It will be interesting to see if others come to the same conclusion.

Builting upon this robust foundation, we have engineered a substantial project, with capabilities far in exceess of the original nanomsg, while still preserving compatibility with the the network protocols that form the backbone of the nanomsg ecosystem, and even a compatible programming interface for nanomsg library users. In addition to compatibility with nanomsg, we find that NNG has greatly increased scalability, reliability, and usability (especially when developing concurrent applications).

NNG also has complete HTTP server and client implementations, support for TLS, and a plethora of other capabilities. Much of this is made possible by a the aforementioned asynchronous I/O framework.

We’ve tried to stay true to the core nanomsg goals about being light-weight, liberally licensed, and implemented in C. (After all, those were the things that drew us to nanomsg in the first place!) In addition we added a couple of new ones. Specifically, reliability, performance, and extensibility (in that order) were added as core goals for the project.

We believe that NNG represents a substantial step forward over other messaging frameworks, and have enjoyed creating it. We hope you find it useful. There is still a lot more we want to do, and future release of NNG will continue to expand it’s capabilities. We’re just getting started.

— Garrett D’Amore, May 30, 2018

Acknowledgements

We would like to thank Janjaap Bos, at Capitar IT Group BV. Without his patronage, neither NNG nor this book would be possible.

We would also like thank Martin Sústrik for creating the original nanomsg project, the foundation upon which all of this work is based.

And certainly not least of all, we would like to thank the various members of the community who have followed and supported the NNG project in so many different ways.

Conventions

Throughout this book there are occasional warnings, notices, and tips. These are visually distinguished as follows:

API Reference

This section is a reference guide for the NNG programming interfaces. It is meant to serve as a reference, rather than as a tutorial.

The material here is organized by major areas of functionality.

Note that unless indicated otherwise, consumers of these interfaces must include the nng/nng.h header file like so:

#include <nng/nng.h>

Sections

• Initialization
• Messages
• Sockets
• Contexts
• Pipes
• Memory
• Time
• URLs
• Synchronization
• Threads
• Logging
• Statistics
• HTTP
• Miscellaneous
• Errors
• ID Map
• Command Options

Initialization & Finalization

This chapter details the function used to initialize the library before first use, and the funtion used to finalize the library and deallocate any resources used by the library.

Initialization

typedef struct {
    int16_t num_task_threads;
    int16_t max_task_threads;
    int16_t num_expire_threads;
    int16_t max_expire_threads;
    int16_t num_poller_threads;
    int16_t max_poller_threads;
    int16_t num_resolver_threads;
} nng_init_params;

extern nng_err nng_init(nng_init_parms *params);

Before using other interfaces in this library, it is necessary to initialize the library. The nng_init function performs this initialization.

The function is idempotent, although on tear down, every call to nng_init must be paired with a call to nng_fini or no resources will be released. This allows for libraries consuming these interfaces to safely initialize and finalize the library without disrupting other consumers in the same process.

Further, only the first call to this function may have a value of params other than NULL. If params is not NULL, and the library has already been intiazed, then nng_init will return NNG_EBUSY.

In some cases it is desirable to tune certain runtime parameters for the library, which can be done by supplying a non-NULL params argument.

Parameters

The individual fields of the nng_init_params structure can be used to adjust certain runtime tunables for the library. There is no guarantee that these tunables are implemented, and applications should not depend upon them for correct operation.

Any member of nng_init_params that is set to zero will be ignored, and any built in default will be used instead for that value.

The following parameters are present:

• num_task_threads and max_task_threads
  Configures the number of threads to use for tasks, which are used principally for completion callbacks. The maximum value can be used to provide an upper limit while still allowing for a dynamically calculated value to be used, as long as it does not exceeed the maximum.

• num_expire_threads and max_expire_threads
  Configures the number of threads used for expiring operations. Using a larger value will reduce contention on some common locks, and may improve performance.

• num_poller_threads and max_poller_threads
  Configures the number of threads to be used for performing I/O. Not all configurations support changing these values.

• num_resolver_threads
  Changes the number of threads used for asynchronous DNS look ups.

Finalization

extern void nng_init(nng_init_parms *params);

When the consumer is ready to deallocate any resoures allocated by the library, it should call the nng_fini function. Each call to nng_fini should be paired with an earlier call to nng_init.

After calling nng_fini, the consuming application must not make any other calls to NNG functions, except that it may use nng_init to initialize the application for further use.

Messages

Messages in Scalability Protocols are the fundamental unit of transmission and reception, as these protocols are fundamentally message-oriented.

Messages have a body, containing the application-supplied payload, and a header, containing protocol specific routing and similar related information.

Message Structure

typedef struct nng_msg nng_msg;

The nng_msg structure represents a single message. It carries a body and a header.

Create a Message

int nng_msg_alloc(nng_msg **msgp, size_t size);

The nng_msg_alloc function allocates a new message. It takes a size argument, and returns a message with a preallocated body of that size in the msgp parameter.

If it succeeds, it returns zero, otherwise this function may return NNG_ENOMEM, indicating that insufficient memory is available to allocate a new message.

Destroy a Message

void nng_msg_free(nng_msg *msg);

The nng_msg_free function deallocates a message.

Duplicate a Message

int nng_msg_dup(nng_msg **dup, nng_msg *msg);

The nng_msg_dup function duplicates the message msg, storing a pointer to the new duplicate in dup. This function also returns zero on succes, or NNG_ENOMEM if memory is exhausted.

Message Size and Capacity

size_t nng_msg_capacity(nng_msg *msg);
int nng_msg_realloc(nng_msg *msg, size_t size);
int nng_msg_reserve(nng_msg *msg, size_t capacity);

Messages have a certain amount of pre-reserved space, which may exceed the total size of the message. This allows for content to be added to the message later, without necessarily performing a reallocation.

The nng_msg_capacity function returns the amount of prereserved space. If a message size change is required, and the new size will fit within the capacity reported by this function, then change will be done without a reallocation, and likely without a data copy as well.

The message size may be changed by use of the nng_msg_realloc function. This function will reallocate the underlying memory for the message msg, preserving contents while doing so. If the new size is smaller than the original message, it will truncate the message, but not perform any allocations. If reallocation fails due to insufficient memory, then the original is left intact.

The nng_msg_reserve function ensures that the total message capacity is at least capacity bytes. Use of this function to ensure the total anticipated capacity is present in the message may help prevent many small allocations.

Both nng_msg_realloc and nng_msg_reserve return zero on success, or may return NNG_ENOMEM if insufficient memory exists to preform allocation.

Message Body

void *nng_msg_body(nng_msg *msg);
size_t nng_msg_len(nng_msg *msg);

The body and body length of msg are returned by nng_msg_body and nng_msg_len, respectively.

Clear the Body

void *nng_msg_clear(nng_msg *msg);

The nng_msg_clear simply resets the total message body length to zero, but does not affect the capacity. It does not change the underlying bytes of the message.

Add to Body

int nng_msg_append(nng_msg *msg, const void *val, size_t size);
int nng_msg_append_u16(nng_msg *msg, uint16_t val16);
int nng_msg_append_u32(nng_msg *msg, uint32_t val32);
int nng_msg_append_u64(nng_msg *msg, uint64_t val64);

int nng_msg_insert(nng_msg *msg, const void *val, size_t size);
int nng_msg_insert_u16(nng_msg *msg, uint16_t val16);
int nng_msg_insert_u32(nng_msg *msg, uint32_t val32);
int nng_msg_insert_u64(nng_msg *msg, uint64_t val64);

Appending data to a message body is done by using the nng_msg_append functions. The base nng_msg_append function appends size bytes of untyped data to the end of the message.

Use of the typed versions, ending in suffixes _u16, _u32, and _u64 allows for unsigned integers to be appended directly. The integers are encoded in network byte order, with the most significant byte appearing first. The message body will by two, four, or eight bytes accordingly.

Data may inserted before the rest of the message body by using the nng_msg_insert functions. This will attempt to use “headroom” in the message to avoid a data copy. Otherwise they are like the nng_msg_append functions except that the put the data in front of the messages instead of at the end.

Consume From Body

int nng_msg_chop(nng_msg *msg, size_t size);
int nng_msg_chop_u16(nng_msg *msg, uint16_t *val16);
int nng_msg_chop_u32(nng_msg *msg, uint32_t *val32);
int nng_msg_chop_u64(nng_msg *msg, uint64_t *val64);

int nng_msg_trim(nng_msg *msg, size_t size);
int nng_msg_trim_u16(nng_msg *msg, uint16_t *val16);
int nng_msg_trim_u32(nng_msg *msg, uint32_t *val32);
int nng_msg_trim_u64(nng_msg *msg, uint64_t *val64);

The nng_msg_chop functions remove data from the end of the body of message msg, reducing the message length by either size, or the appropriate value size.

The nng_msg_trim functions remove data from the beginning of the message body of msg. but are otherwise just like the nng_msg_chop functions.

If the message is not big enough to remove requisite amount of bytes, these functions return NNG_EINVAL. Otherwise they return zero.

Additionally, functions with typed suffixes (_u16, _u32, _u64) decode the data and return it through the appropriate val pointer.

The data is assumed to have been in network byte order in the message, but is returned in the native machine byte order. The appropriate number of bytes is consumed for each of these types, so two bytes for _u16, four bytes for _u32, and eight bytes for _u64.

Message Header

void *nng_msg_header(nng_msg *msg);
size_t nng_msg_header_len(nng_msg *msg);

The header and header length of msg are returned by nng_msg_header and nng_msg_header_len, respectively.

The message headers are generally intended for limited use, to store protocol headers.

Clear the Header

void *nng_msg_header_clear(nng_msg *msg);

The nng_msg_header_clear simply resets the total message header length to zero.

Append or Insert Header

int nng_msg_header_append(nng_msg *msg, const void *val, size_t size);
int nng_msg_header_append_u16(nng_msg *msg, uint16_t val16);
int nng_msg_header_append_u32(nng_msg *msg, uint32_t val32);
int nng_msg_header_append_u64(nng_msg *msg, uint64_t val64);

int nng_msg_header_insert(nng_msg *msg, const void *val, size_t size);
int nng_msg_header_insert_u16(nng_msg *msg, uint16_t val16);
int nng_msg_header_insert_u32(nng_msg *msg, uint32_t val32);
int nng_msg_header_insert_u64(nng_msg *msg, uint64_t val64);

Appending data to a message header is done by using the nng_msg_header_append functions, and inserting data in the header is done using the nng_msg_header_insert functions.

These functions act just like the nng_msg_append and nng_msg_insert functions, except that they operate on the message header rather than the message body.

Consume from Header

int nng_msg_header_chop(nng_msg *msg, size_t size);
int nng_msg_header_chop_u16(nng_msg *msg, uint16_t *val16);
int nng_msg_header_chop_u32(nng_msg *msg, uint32_t *val32);
int nng_msg_header_chop_u64(nng_msg *msg, uint64_t *val64);

int nng_msg_header_trim(nng_msg *msg, size_t size);
int nng_msg_header_trim_u16(nng_msg *msg, uint16_t *val16);
int nng_msg_header_trim_u32(nng_msg *msg, uint32_t *val32);
int nng_msg_header_trim_u64(nng_msg *msg, uint64_t *val64);

The nng_msg_header_trim functions remove data from the beginning of the message header, and the nng_msg_header_chop functions remove data from the end of the message header.

These functions act just like the nng_msg_trim and nng_msg_chop functions, except that they operate the message header rather than the message body.

Message Pipe

nng_pipe nng_msg_get_pipe(nng_msg *msg);
void nng_msg_get_pipe(nng_msg *msg, nng_pipe p);

The nng_msg_set_pipe function sets the pipe associated with msg to p. This is most often useful when used with protocols that support directing a message to a specific peer. For example the PAIR version 1 protocol can do this when NNG_OPT_PAIR1_POLY mode is set.

The nng_msg_get_pipe function returns the pipe that was previously set on the message m, either directly by the application, or when the message was received by the protocol.

Examples

Example 1: Preparing a message for use

#include <nng/nng.h>

nng_msg *m;
if (nng_msg_alloc(&m, strlen("content") + 1) != 0) {
  // handle error
}
strcpy(nng_msg_body(m), "content");

Example 2: Preallocating message content

if (nng_msg_alloc(&m, 1024) != 0) {
  // handle error
}
while ((val64 = next_datum()) != 0) P
  if (nng_msg_append_u64(m, val64) != 0) {
      // handle error
  }
}

Sockets

Sockets in Scalability Protocols provide the handle for communication between peers. Sockets also encapsulate protocol specific semantics, such as filtering subscriptions, or automatically retrying requests.

Socket Structure

#define NNG_SOCKET_INITIALIZER // opaque value

typedef struct nng_socket_s nng_socket;

The nng_socket structure represents socket. This is a handle, and the members of it are opaque. However, unlike a pointer, it is usually passed by value.

A socket may be initialized statically with the NNG_SOCKET_INITIALIZER macro, to ensure that it cannot be confused with a valid open socket.

Socket Identity

int nng_socket_id(nng_socket s);
int nng_socket_raw(nng_socket s, bool *raw);
int nng_socket_proto_id(nng_socket s, uint16_t *proto);
int nng_socket_peer_id(nng_socket s, uint16_t *proto);
int nng_socket_proto_name(nng_socket s, const char **name);
int nng_socket_peer_name(nng_socket s, const char **name);

These functions are used to provide fundamental information about the socket s. Most applications will not need to use these functions.

The nng_socket_id function returns the numeric id, which will be a non-negative value, associated with the socket. If the socket is uninitialized (has never been opened), then the return value may be -1.

The nng_socket_proto_id and nng_socket_peer_id functions provide the 16-bit protocol identifier for the socket’s protocol, and of the protocol peers will use when communicating with the socket.

The nng_socket_proto_name and nng_socket_peer_name functions provide the ASCII names of the socket’s protocol, and of the protocol peers of the socket use. The value stored in name is a fixed string located in program text, and must not be freed or altered. It is guaranteed to remain valid while this library is present.

The nng_socket_raw function determines whether the socket is in raw mode or not, storing true in raw if it is, or false if it is not.

Opening a Socket

int nng_bus0_open(nng_socket *s);
int nng_pub0_open(nng_socket *s);
int nng_pull0_open(nng_socket *s);
int nng_push0_open(nng_socket *s);
int nng_rep0_open(nng_socket *s);
int nng_req0_open(nng_socket *s);
int nng_respondent0_open(nng_socket *s);
int nng_sub0_open(nng_socket *s);
int nng_surveyor0_open(nng_socket *s);

These functions open a socket, returning it in s. The constructors for sockets are protocol specific so please refer to protocol documentation for more specific information.

The following functions open a socket in normal mode:

• nng_bus0_open - BUS version 0
• nng_pair0_open - PAIR version 0
• nng_pair1_open - PAIR version 1
• nng_pair1_open_poly - PAIR version 1, polyamorous mode
• nng_pub0_open - PUB version 0
• nng_pull0_open - PULL version 0
• nng_push0_open - PUSH version 0
• nng_rep0_open - REP version 0
• nng_req0_open - REQ version 0
• nng_respondent0_open - RESPONDENT version 0
• nng_sub0_open - SUB version 0
• nng_surveyor0_open - SURVEYOR version 0

Raw Mode Sockets

int nng_bus0_open_raw(nng_socket *s);
int nng_pub0_open_raw(nng_socket *s);
int nng_pull0_open_raw(nng_socket *s);
int nng_push0_open_raw(nng_socket *s);
int nng_rep0_open_raw(nng_socket *s);
int nng_req0_open_raw(nng_socket *s);
int nng_respondent0_open_raw(nng_socket *s);
int nng_sub0_open_raw(nng_socket *s);
int nng_surveyor0_open_raw(nng_socket *s);

Raw mode sockets are used in circumstances when the application needs direct access to the message headers to control the protocol details.

Such sockets require greater sophistication on the part of the application to use, as the application must process the protocol headers specifically. The details of the protocol headers, and requirements, are described in the protocol documentation for each protocol.

Raw mode sockets do not have any kind of state machine associated with them, as all of the protocol specific processing must be performed by the application.

The following functions open a socket in raw mode:

• nng_bus0_open_raw - BUS version 0, raw mode
• nng_pair0_open_raw - PAIR version 0, raw mode
• nng_pair1_open_raw - PAIR version 1, raw mode
• nng_pub0_open_raw - PUB version 0, raw mode
• nng_pull0_open_raw - PULL version 0, raw mode
• nng_push0_open_raw - PUSH version 0, raw mode
• nng_rep0_open_raw - REP version 0, raw mode
• nng_req0_open_raw - REP version 0, raw mode
• nng_respondent0_open_raw - RESPONDENT version 0, raw mode
• nng_sub0_open_raw - SUB version 0, raw mode
• nng_surveyor0_open_raw - SURVEYOR version 0, raw mode

Closing a Socket

int nng_socket_close(nng_socket s);

The nng_socket_close function closes a socket, releasing all resources associated with it. Any operations that are in progress will be terminated with a result of NNG_ECLOSED.

Sending Messages

int nng_send(nng_socket s, void *data, size_t size, int flags);
int nng_sendmsg(nng_socket s, nng_msg *msg, int flags);
void nng_socket_send(nng_socket s, nng_aio *aio);

These functions (nng_send, nng_sendmsg, and nng_socket_send) send messages over the socket s. The differences in their behaviors are as follows.

nng_send

The nng_send function is the simplest to use, but is the least efficient. It sends the content in data, as a message of size bytes. The flags is a bit mask made up of zero or more of the following values:

• NNG_FLAG_NONBLOCK: If the socket cannot accept more data at this time, it does not block, but returns immediately with a status of NNG_EAGAIN. If this flag is absent, the function will wait until data can be sent.

nng_sendmsg

The nng_sendmsg function sends the msg over the socket s.

If this function returns zero, then the socket will dispose of msg when the transmission is complete. If the function returns a non-zero status, then the call retains the responsibility for disposing of msg.

The flags can contain the value NNG_FLAG_NONBLOCK, indicating that the function should not wait if the socket cannot accept more data for sending. In such a case, it will return NNG_EAGAIN.

nng_socket_send

The nng_socket_send function sends a message asynchronously, using the nng_aio aio, over the socket s. The message to send must have been set on aio using the nng_aio_set_msg function.

If the operation completes successfully, then the socket will have disposed of the message. However, if it fails, then callback of aio should arrange for a final disposition of the message. (The message can be retrieved from aio with nng_aio_get_msg.)

Note that callback associated with aio may be called before the message is finally delivered to the recipient. For example, the message may be sitting in queue, or located in TCP buffers, or even in flight.

Receiving Messages

int nng_recv(nng_socket s, void *data, size_t *sizep, int flags);
int nng_recvmsg(nng_socket s, nng_msg **msgp, int flags);
void nng_socket_recv(nng_socket s, nng_aio *aio);

These functions (nng_recv, nng_recvmsg, and nng_socket_recv) receive messages over the socket s. The differences in their behaviors are as follows.

nng_recv

The nng_recv function is the simplest to use, but is the least efficient. It receives the content in data, as a message size (in bytes) of up to the value stored in sizep.

Upon success, the size of the message received will be stored in sizep.

The flags is a bit mask made up of zero or more of the following values:

• NNG_FLAG_NONBLOCK: If the socket has no messages pending for reception at this time, it does not block, but returns immediately with a status of NNG_EAGAIN. If this flag is absent, the function will wait until data can be received.

nng_recvmsg

The nng_recvmsg function receives a message and stores a pointer to the nng_msg for that message in msgp.

The flags can contain the value NNG_FLAG_NONBLOCK, indicating that the function should not wait if the socket has no messages available to receive. In such a case, it will return NNG_EAGAIN.

nng_socket_recv

The nng_socket_send function receives a message asynchronously, using the nng_aio aio, over the socket s. On success, the received message can be retrieved from the aio using the nng_aio_get_msg function.

Socket Options

int nng_socket_get_bool(nng_socket s, const char *opt, bool *valp);
int nng_socket_get_int(nng_socket s, const char *opt, int *valp);
int nng_socket_get_ms(nng_socket s, const char *opt, nng_duration *valp);
int nng_socket_get_size(nng_socket s, const char *opt, size_t *valp);

int nng_socket_set_bool(nng_socket s, const char *opt, int val);
int nng_socket_set_int(nng_socket s, const char *opt, int val);
int nng_socket_set_ms(nng_socket s, const char *opt, nng_duration val);
int nng_socket_set_size(nng_socket s, const char *opt, size_t val);

Protocols usually have protocol specific behaviors that can be adjusted via options.

These functions are used to retrieve or change the value of an option named opt from the context ctx. The nng_socket_get_ functions retrieve the value from the socket s, and store it in the location valp references. The nng_socket_set_ functions change the value for the socket s, taking it from val.

These functions access an option as a specific type. The protocol documentation will have details about which options are available, whether they can be read or written, and the appropriate type to use.

Common Options

The following options are available for many protocols, and always use the same types and semantics described below.

Polling Socket Events

int nng_socket_get_recv_poll_fd(nng_socket s, int *fdp);
int nng_socket_get_send_poll_fd(nng_socket s, int *fdp);

Sometimes it is necessary to integrate a socket into a poll or select driven event loop. (Or, on Linux, epoll, or on BSD derived systems like macOS kqueue).

For these occasions, a suitable file descriptor for polling is provided by these two functions.

The nng_socket_get_recv_poll_fd function obtains a file descriptor that will poll as readable when a message is ready for receiving for the socket.

The nng_socket_get_send_poll_fd function obtains a file descriptor that will poll as readable when the socket can accept a message for sending.

These file descriptors should only be polled for readability, and no other operation performed on them. The socket will read from, or write to, these file descriptors to provide a level-signaled behavior automatically.

Additionally the socket will close these file descriptors when the socket itself is closed.

These functions replace the NNG_OPT_SENDFD and NNG_OPT_RECVFD socket options that were available in previous versions of NNG.

Examples

Example 1: Initializing a Socket

nng_socket s = NNG_SOCKET_INITIALIZER;

Example 2: Publishing a Timestamp

This example demonstrates the use of nng_aio, nng_socket_send, and nng_sleep_aio to build a service that publishes a timestamp at one second intervals. Error handling is elided for the sake of clarity.

#include <stdlib.h>
#include <stdio.h>
#include <nng/nng.h>
#include <nng/protocol/pubsub0/pub.h>

struct state {
    nng_socket s;
    bool sleeping;
    nng_aio *aio;
};

static struct state state;

void callback(void *arg) {
    nng_msg *msg;
    nng_time now;
    struct state *state = arg;
    if (nng_aio_result(state->aio) != 0) {
        fprintf(stderr, "Error %s occurred", nng_strerror(nng_aio_result(state->aio)));
        return; // terminate the callback loop
    }
    if (state->sleeping) {
        state->sleeping = false;
        nng_msg_alloc(&msg, sizeof (nng_time));
        now = nng_clock();
        nng_msg_append(msg, &now, sizeof (now)); // note: native endian
        nng_aio_set_msg(state->aio, msg);
        nng_socket_send(state->s, state->aio);
    } else {
        state->sleeping = true;
        nng_sleep_aio(1000, state->aio); // 1000 ms == 1 second
    }
}

int main(int argc, char **argv) {
    const char *url = argv[1]; // should check this

    nng_aio_alloc(&state.aio, NULL, NULL);
    nng_pub0_open(&state.s);
    nng_listen(state.s, url, NULL, 0);
    state.sleeping = 0;
    nng_sleep_aio(1, state.aio); // kick it off right away
    for(;;) {
        nng_msleep(0x7FFFFFFF); // infinite, could use pause or sigsuspend
    }
}

Example 3: Watching a Periodic Timestamp

This example demonstrates the use of nng_aio, nng_socket_recv, to build a client to watch for messages received from the service created in Example 2. Error handling is elided for the sake of clarity.

#include <stdlib.h>
#include <stdio.h>
#include <string.h>>
#include <nng/nng.h>
#include <nng/protocol/pubsub0/sub.h>

struct state {
    nng_socket s;
    nng_aio *aio;
};

static struct state state;

void callback(void *arg) {
    nng_msg *msg;
    nng_time now;
    struct state *state = arg;
    if (nng_aio_result(state->aio) != 0) {
        fprintf(stderr, "Error %s occurred", nng_strerror(nng_aio_result(state->aio)));
        return; // terminate the callback loop
    }
    msg = nng_aio_get_msg(state->aio);
    memcpy(&now, nng_msg_body(msg), sizeof (now)); // should check the length!
    printf("Timestamp is %lu\n", (unsigned long)now);
    nng_msg_free(msg);
    nng_aio_set_msg(state->aio, NULL);
    nng_socket_recv(state->s, state->aio);
}

int main(int argc, char **argv) {
    const char *url = argv[1]; // should check this

    nng_aio_alloc(&state.aio, NULL, NULL);
    nng_sub0_open(&state.s);
    nng_sub0_socket_subscribe(state.s, NULL, 0); // subscribe to everything
    nng_dial(state.s, url, NULL, 0);
    nng_socket_recv(state.s, state.aio); // kick it off right away
    for(;;) {
        nng_msleep(0x7FFFFFFF); // infinite, could use pause or sigsuspend
    }
}

Contexts

Contexts in Scalability Protocols provide for isolation of protocol-specific state machines and associated data, allowing multiple concurrent state machines (or transactions) to coexist on a single socket.

For example, a REP server may wish to allow many requests to be serviced concurrently, even though some jobs may take significant time to process. Contexts provide for this ability.

Not all protocols have contexts, because many protocols simply have no state to manage. The following protocols support contexts:

• REP
• REQ
• RESPONDENT
• SURVEYOR
• SUB

For these protocols, the socket will also have a single, default, context that is used when performing send or receive operations on the socket directly.

Other protocols are stateless, at least with respect to message processing, and have no use for contexts. For the same reason, raw mode sockets do not support contexts.

One drawback of contexts is that they cannot be used with file descriptor polling using nng_socket_get_recv_poll_fd or nng_socket_get_send_poll_fd.

Context Structure

#define NNG_CTX_INITIALIZER // opaque value

typedef struct nng_ctx_s nng_ctx;

The nng_ctx structure represents context. This is a handle, and the members of it are opaque. However, unlike a pointer, it is passed by value.

A context may be initialized statically with the NNG_CTX_INITIALIZER macro, to ensure that it cannot be confused with a valid open context.

Creating a Context

int nng_ctx_open(nng_ctx *ctxp, nng_socket s);

The nng_ctx_open function creates a separate context to be used with the socket s, and returns it at the location pointed by ctxp.

Context Identity

int nng_ctx_id(nng_ctx c);

The nng_ctx_id function returns a positive identifier for the context c if it is valid. Otherwise it returns -1.

A context is considered valid if it was ever opened with nng_ctx_open function. Contexts that are allocated on the stack or statically should be initialized with the macro NNG_CTX_INITIALIZER to ensure that they cannot be confused with a valid context before they are opened.

Closing a Context

int nng_ctx_close(nng_ctx ctx);

The nng_ctx_close function closes the context ctx. Messages that have been submitted for sending may be flushed or delivered, depending upon the transport.

Further attempts to use the context after this call returns will result in NNG_ECLOSED. Threads waiting for operations on the context when this call is executed may also return with an NNG_ECLOSED result.

Sending Messages

int nng_ctx_sendmsg(nng_ctx ctx, nng_msg *msg, int flags);
void nng_ctx_send(nng_ctx ctx, nng_aio *aio);

These functions (nng_ctx_sendmsg and nng_ctx_send) send messages over the socket s. The differences in their behaviors are as follows.

nng_ctx_sendmsg

The nng_ctx_sendmsg function sends the msg over the context ctx.

If this function returns zero, then the socket will dispose of msg when the transmission is complete. If the function returns a non-zero status, then the call retains the responsibility for disposing of msg.

The flags can contain the value NNG_FLAG_NONBLOCK, indicating that the function should not wait if the socket cannot accept more data for sending. In such a case, it will return NNG_EAGAIN.

nng_ctx_send

The nng_ctx_send function sends a message asynchronously, using the nng_aio aio, over the context ctx. The message to send must have been set on aio using the nng_aio_set_msg function.

If the operation completes successfully, then the context will have disposed of the message. However, if it fails, then callback of aio should arrange for a final disposition of the message. (The message can be retrieved from aio with nng_aio_get_msg.)

Note that callback associated with aio may be called before the message is finally delivered to the recipient. For example, the message may be sitting in queue, or located in TCP buffers, or even in flight.

Receiving Messages

int nng_ctx_recvmsg(nng_ctx ctx, nng_msg **msgp, int flags);
void nng_ctx_recv(nng_ctx ctx, nng_aio *aio);

These functions (, nng_ctx_recvmsg and nng_ctx_recv) receive messages over the context ctx. The differences in their behaviors are as follows.

nng_recvmsg

The nng_ctx_recvmsg function receives a message and stores a pointer to the nng_msg for that message in msgp.

The flags can contain the value NNG_FLAG_NONBLOCK, indicating that the function should not wait if the socket has no messages available to receive. In such a case, it will return NNG_EAGAIN.

nng_socket_recv

The nng_ctx_send function receives a message asynchronously, using the nng_aio aio, over the context ctx. On success, the received message can be retrieved from the aio using the nng_aio_get_msg function.

Context Options

int nng_ctx_get_bool(nng_ctx ctx, const char *opt, bool *valp);
int nng_ctx_get_int(nng_ctx ctx, const char *opt, int *valp);
int nng_ctx_get_ms(nng_ctx ctx, const char *opt, nng_duration *valp);
int nng_ctx_get_size(nng_ctx ctx, const char *opt, size_t *valp);

int nng_ctx_set_bool(nng_ctx ctx, const char *opt, int val);
int nng_ctx_set_int(nng_ctx ctx, const char *opt, int val);
int nng_ctx_set_ms(nng_ctx ctx, const char *opt, nng_duration val);
int nng_ctx_set_size(nng_ctx ctx, const char *opt, size_t val);

Some protocols support certain options that affect the behavior of a specific context. For example, most protocols will let you set the defaults timeouts associated with send or receive separately for different contexts.

These functions are used to retrieve or change the value of an option named opt from the context ctx. The nng_ctx_get_ functions retrieve the value from ctx, and store it in the location valp references. The nng_ctx_set_ functions change the value for the ctx, taking it from val.

These functions access an option as a specific type. The protocol documentation will have details about which options are available for contexts, whether they can be read or written, and the appropriate type to use.

Examples

These examples show building blocks for a concurrent service based on contexts.

Example 1: Context Echo Server

The following program fragment demonstrates the use of contexts to implement a concurrent REP service that simply echos messages back to the sender.

struct echo_context {
    nng_ctx ctx;
    nng_aio *aio;
    enum { INIT, RECV, SEND } state;
};

void
echo(void *arg)
{
    struct echo_context *ec = arg;

    switch (ec->state) {
    case INIT:
        ec->state = RECV;
        nng_ctx_recv(ec->ctx, ec->aio);
        return;
    case RECV:
        if (nng_aio_result(ec->aio) != 0) {
            // ... handle error
        }
        // We reuse the message on the ec->aio
        ec->state = SEND;
        nng_ctx_send(ec->ctx, ec->aio);
        return;
    case SEND:
        if (nng_aio_result(ec->aio) != 0) {
            // ... handle error
        }
        ec->state = RECV;
        nng_ctx_recv(ec->ctx, ec->aio);
        return;
    }
}

Example 2: Starting the Echo Service

Given the above fragment, the following example shows setting up the service. It assumes that the socket has already been created and any transports set up as well with functions such as nng_dial or nng_listen.

#define CONCURRENCY 1024

static struct echo_context ecs[CONCURRENCY];

void
start_echo_service(nng_socket rep_socket)
{
    for (int i = 0; i < CONCURRENCY; i++) {
        // error checks elided for clarity
        nng_ctx_open(&ecs[i].ctx, rep_socket);
        nng_aio_alloc(&ecs[i].aio, echo, ecs+i);
        ecs[i].state = INIT;
        echo(ecs+i); // start it running
    }
}

Pipes

typedef struct nng_pipe_s nng_pipe;

An nng_pipe is a handle to a pipe object, which can be thought of as a single connection. (In most cases this is actually the case – the pipe is an abstraction for a single TCP or IPC connection.) Pipes are associated with either the listener or dialer that created them, and therefore are also automatically associated with a single socket.

Pipe objects are created by dialers and and listeners.

Initialization

A pipe may be initialized using the macro NNG_PIPE_INITIALIZER before it is opened, to prevent confusion with valid open pipes.

For example:

nng_pipe p = NNG_PIPE_INITIALIZER;

Pipe Identity

int nng_pipe_id(nng_pipe p);

The nng_pipe_id function returns a positive identifier for the pipe p, if it is valid. Otherwise it returns -1.

Closing a Pipe

nng_err nng_pipe_close(nng_pipe p);

The nng_pipe_close function closes the supplied pipe, p. Messages that have been submitted for sending may be flushed or delivered, depending upon the transport.

Further attempts to use the pipe after this call returns will result in NNG_ECLOSED.

Pipe Creator

nng_dialer nng_pipe_dialer(nng_pipe p);
nng_listener nng_pipe_listener(nng_pipe p);
nng_socket nng_pipe_socket(nng_pipe p);

These functions return the socket, dialer, or listener that created or owns the pipe.

If the pipe was does not have an associated dialer or listener, then the associated will return [NNG_DIALER_INITIALIZER] or [NNG_LISTENER_INITIALIZER], as appropriate, and either [nng_dialer_id] or [nng_listener_id] for the returned object will return -1.

Pipe Options

nng_err nng_pipe_get_bool(nng_pipe p, const char *opt, bool *valp);
nng_err nng_pipe_get_int(nng_pipe p, const char *opt, int *valp);
nng_err nng_pipe_get_ms(nng_pipe p, const char *opt, nng_duration *valp);
nng_err nng_pipe_get_size(nng_pipe p, const char *opt, size_t *valp);
nng_err nng_pipe_get_addr(nng_pipe p, const char *opt, nng_sockaddr *valp);
nng_err nng_pipe_get_string(nng_pipe p, const char *opt, char **valp);

These functions are used to obtain value of an option named opt from the pipe p, and store it in the location referenced by valp.

These functions access an option as a specific type. The transport layer will have details about which options are available, and which type they may be accessed using.

In the case of nng_pipe_get_string, the string is created as if by nng_strdup, and must be freed by the caller using nng_strfree when no longer needed.

Pipe Notifications

typedef enum {
        NNG_PIPE_EV_ADD_PRE,
        NNG_PIPE_EV_ADD_POST,
        NNG_PIPE_EV_REM_POST,
} nng_pipe_ev;

typedef void (*nng_pipe_cb)(nng_pipe, nng_pipe_ev, void *);

nng_err nng_pipe_notify(nng_socket s, nng_pipe_ev ev, nng_pipe_cb cb, void *arg);

The nng_pipe_notify function registers the callback function cb to be called whenever the pipe event specified by ev occurs on the socket s. The callback cb will be passed arg as its final argument.

A different callback may be supplied for each event. Each event may have at most one callback registered. Registering a callback implicitly unregisters any previously registered.

The following pipe events are supported:

Memory

Managing memory and allocations is something that every C program has to deal with. In the case of NNG, it can be more complicated because the underlying platform code can provide different allocators that might not be compatible with the use system allocator used by malloc and free.

Allocate Memory

void *nng_alloc(size_t size);

The nng_alloc function allocates a contiguous memory region of at least size bytes, and returns a pointer to it. The memory will be 64-bit aligned. Note that the memory may have random data in it, just like with malloc.

If memory cannot be allocated for any reason, then NULL will be returned. Applications that experience this should treat this like NNG_ENOMEM.

Memory returned by nng_alloc should be freed when no longer needed using nng_free.

Deallocate Memory

void nng_free(void *ptr, size_t size);

The nng_free function deallocates memory previously allocated by nng_alloc.

The size argument must exactly match the size argument that was supplied to nng_alloc when the memory was allocated.

Duplicate String

char *nng_strdup(const char *src);

The nng_strdup duplicates the string src and returns it.

This is logically equivalent to using nng_alloc to allocate a region of memory of strlen(s) + 1 bytes, and then using strcpy to copy the string into the destination before returning it.

The returned string should be deallocated with nng_strfree, or may be deallocated using the nng_free using the length of the returned string plus one (for the NUL terminating byte).

Free String

void nng_strfree(char *str);

The nng_strfree function is a convenience function that can be used to deallocate strings allocated with nng_strdup.

It is effectively the same as nng_free(strlen(str) + 1).

Time

NNG supports has support for time in the form of access to a system clock, and supporting timeouts for certain operations.

Time Type

typedef uint64_t nng_time;

The nng_time type is used to represent a clock offset from a common base time, measured in milliseconds.

The reference, or zero value, is some arbitrary point in time, most often sytem boot, but can be process start time or any other convenient reference.

All threads within a process will use the same reference time, but be aware that different processes may use a different reference time.

Duration Type

typedef int64_t nng_duration;

#define NNG_DURATION_INFINITE (-1)
#define NNG_DURATION_DEFAULT  (-2)
#define NNG_DURATION_ZERO     (0)

The nng_duration time measures a time duration in milliseconds. Normally durations are positive, but some specific negative values are reserved.

• NNG_DURATION_INFINITE: The duration essentially means forever. This is most often used with a timeout to indicate that there is no timeout, the operation should wait until it is complete, even forever.

• NNG_DURATION_DEFAULT: This special value is used in some circumstances to prevent overriding a default timeout. Some operations have a default maximum time, and this value means that the previously established default should be used. The precise meaning is context-specific.

• NNG_DURATION_ZERO: A zero length duration is used to performan an immediate poll.

Get the Current Time

nng_time nng_clock(void);

The nng_clock function returns the number of elapsed milliseconds since some arbitrary time in the past. The resolution of the clock depends on the underlying timing facilities of the system. This function may be used for timing, but applications should not expect very fine-grained values.

Wait for Duration

void nng_msleep(nng_duration msec);

The nng_msleep function blocks the calling thread for at least the specified number of milliseconds.

Wait Asynchronously

void nng_sleep_aio(nng_duration msec, nng_aio *aio);

It is possible to wait as the action on an nng_aio, which in effect acts like scheduling a callback to run after a specified period of time.

The nng_sleep_aio function provides this capability. After msec milliseconds have passed, then aio’s callback will be executed. If this sleep waits without interruption, and then completes, the result from nng_aio_result will be zero.

See Also

Asynchronous Operations, Synchronization

URLs

Universal Resource Locators, or URLs for short, are a standardized way of representing a network resource, defined in RFC 1738, and RFC 3968.

In Scalability Protocols, this concept is extended, although it includes schemes that are not part of the IETF standards.

URL Structure

typedef struct nng_url nng_url;

const char *nng_url_scheme(const nng_url *url);
const char *nng_url_userinfo(const nng_url *url);
const char *nng_url_hostname(const nng_url *url);
uint32_t    nng_url_port(const nng_url *url);
const char *nng_url_path(const nng_url *url);
const char *nng_url_query(const nng_url *url);
const char *nng_url_fragment(const nng_url *url):

URL Fields

The nng_url_scheme function returns the scheme, without any colons or slashes. Values are lower case strings, like “http” or “inproc” or “tls+tcp4”.

The nng_url_userinfo function returns a string corresponding to the user component of a URL (the part before any @ sign) if such a component is present, otherwise it returns NULL.

The nng_url_hostname function returns a hostname (which might actually be an IP address) from the URL, if the URL corresponds to a scheme that uses hostnames (like “http” or “tcp”). If the URL does not (for example “inproc” or “ipc” URLs) then it returns NULL.

The nng_url_port function returns the TCP or UDP port number if the URL corresponds to a protocol based on TCP or UDP. It returns zero otherwise. Note that the port number might not have been explicitly specified in the URL. For example, the port number associated with “http://www.example.com” is 80, which is the standard port number for HTTP.

The nng_url_path function returns the path component of the URL. This will always be non-NULL, but it may be empty.

The nng_url_query and nng_url_fragment functions return the query-information (the part following a ‘?’) and fragment (the part following a ‘#’) if those components are present, or NULL if they are not. The returned string will not include the leading ‘?’ or ‘#’ characters.

Note that any strings returned by these functions are only valid until url is freed with nng_url_free.

Format a URL

int nng_url_sprintf(char *buf, size_t bufsz, const nng_url *url);

The nng_url_sprintf function formats the url to the buf, which must have bufsz bytes of free space associated with it.

This function returns the number of bytes formatted to buf, excludng the terminating zero byte, or if bufsz is too small, then it returns the number of bytes that would have been formatted if there was sufficient space. The semantics are similar to the snprintf function from C99.

Parse a URL

int nng_url_parse(nng_url **urlp, const char *str);

The nng_url_parse function parses a URL string (in str), and creates a dynamically allocated nng_url, returning it in urlp.

Clone a URL

int nng_url_clone(nng_url **dup, const nng_url *url);

The nng_url_clone function creates a copy of url, and returns it in dup.

Destroy a URL

void nng_url_free(nng_url *url);

The nng_url_free function destroy an nng_url object created with either nng_url_parse or nng_url_free.

This is the only correct way to destroy an nng_url object.

Update a URL Port

void nng_url_resolve_port(nng_url *url, uint32_t port);

In order to use dynamic port allocation, some transports and schemes allow a port number of zero to be specified. When this is done, the system will allocate a free port when the port is bound while starting a listener. Once this is done, the port number may need to be updated so that the URL can be used to configure a client. The nng_url_resolve_port function updates such a URL with a new port. This will have no effect if the URL already has a non-zero port number.

See Also

More information about Universal Resource Locators can be found in RFC 3986.

Asynchronous Operations

In order to obtain significant scalability, with low-latency, and minimal overheads, NNG supports performing operations asynchronously.

One way that applications can perform work asynchronously and concurrently is by using threads, but threads carry significant resource overheads and frequently there are limits on the number that can easily be created.

Additionally, with most network applications, the flow of execution will spend the bulk of its time waiting for traffic from a peer.

For these kinds of applications, it is far more efficient to use asynchronous operations using the mechanisms described in this chapter.

Asynchronous I/O Handle

typedef struct nng_aio nng_aio;

An nng_aio is an opaque structure used in conjunction with asynchronous I/O operations. Every asynchronous operation uses one of these structures, each of which can only be used with a single operation at a time.

Asynchronous operations are performed without blocking calling application threads. Instead the application registers a callback function to be executed when the operation is complete (whether successfully or not). This callback will be executed exactly once.

The asynchronous I/O framework also supports cancellation of operations that are already in progress as well setting a maximum timeout for them to complete.

It is also possible to initiate an asynchronous operation, and wait for it to complete, creating a synchronous flow from an asynchronous one.

Create Handle

nng_err nng_aio_alloc(nng_aio **aiop, void (*callb)(void *), void *arg);

The nng_aio_alloc function creates an nng_aio object, with the callback callb taking the argument arg, and returns it in aiop.

If this succeeds, the function returns zero. Otherwise it may return NNG_ENOMEM.

Destroy Handle

void nng_aio_free(nng_aio *aio);
void nng_aio_reap(nng_aio *aio);

The nng_aio_free handle destroys the handle aio, waiting for any operations and associated callbacks to complete before doing so.

The nng_aio_reap handle destroys the handle aio asynchronously, using a reaper thread to do so. It does not wait for the object to be destroyed. Thus this function is safe to call from aio’s own callback.

Cancellation

void nng_aio_abort(nng_aio *aio, nng_err err);
void nng_aio_cancel(nng_aio *aio);
void nng_aio_stop(nng_aio *aio);

These functions are used to stop a previously submitted asynchronous I/O operation. The operation may be canceled, or may continue to completion. If no operation is in progress (perhaps because it has already completed), then these operations have no effect. If the operation is successfully canceled or aborted, then the callback will still be called.

The nng_aio_abort function aborts the operation associated with aio and returns immediately without waiting. If cancellation was successful, then nng_aio_result will return err.

The nng_aio_cancel function acts like nng_aio_abort, but uses the error code NNG_ECANCELED.

The nng_aio_stop function aborts the aio operation with NNG_ESTOPPED, and then waits the operation and any associated callback to complete. This function also marks aio itself permanently stopped, so that any new operations scheduled by I/O providers using nng_aio_start return false. Thus this function should be used to teardown operations.

Set Timeout

void nng_aio_set_timeout(nng_aio *aio, nng_duration timeout);
void nng_aio_set_expire(nng_aio *aio, nng_time expiration);

The nng_aio_set_timeout function sets a timeout for the asynchronous operation associated with aio. This causes a timer to be started when the operation is actually started. If the timer expires before the operation is completed, then it is aborted with an error of NNG_ETIMEDOUT. The timeout duration is specified as a relative number of milliseconds.

If the timeout is NNG_DURATION_INFINITE, then no timeout is used. If the timeout is NNG_DURATION_DEFAULT, then a “default” or socket-specific timeout is used. (This is frequently the same as NNG_DURATION_INFINITE.)

The nng_aio_set_expire function is similar to nng_aio_set_timeout, but sets an expiration time based on the system clock. The expiration time is a clock timestamp, such as would be returned by nng_clock.

Wait for Completion

void nng_aio_wait(nng_aio *aio);

The nng_aio_wait function waits for an asynchronous I/O operation to complete. If the operation has not been started, or has already completed, then it returns immediately.

If a callback was set with aio when it was allocated, then this function will not be called until the callback has completed.

Test for Completion

bool nng_aio_busy(nng_aio *aio);

The nng_aio_busy function returns true if the aio is currently busy performing an operation or is executing a completion callback. Otherwise it return false. This is the same test used internally by nng_aio_wait.

Result of Operation

nng_err nng_aio_result(nng_aio *aio);
size_t nng_aio_count(nng_aio *aio);

The nng_aio_result function returns the result of the operation associated with the handle aio. If the operation was successful, then 0 is returned. Otherwise a non-zero error code, such as NNG_ECANCELED or NNG_ETIMEDOUT, is returned.

For operations that transfer data, nng_aio_count returns the number of bytes transferred by the operation associated with the handle aio. Operations that do not transfer data, or do not keep a count, may return zero for this function.

Messages

nng_msg *nng_aio_get_msg(nng_aio *aio);
void nng_aio_set_msg(nng_aio *aio, nng_msg *msg);

The nng_aio_get_msg and nng_aio_set_msg functions retrieve and store a message in aio. For example, if a function to receive data is called, that function can generally be expected to store a message on the asssociated aio, for the application to retrieve with nng_aio_get_msg. Conversely an application desiring to send a message msg will store it in the aio using nng_aio_set_msg. The function implementing the send operation will retrieve the message and arrange for it to be sent.

Message Ownership

For send or transmit operations, the rule of thumb is that implementation of the operation is responsible for taking ownership of the message (and releasing resources when it is complete), if it will return success. If the operation will end in error, then the message will be retained and it is the consuming application’s responsibility to dispose of the message. This allows an application the opportunity to reuse the message to try again, if it so desires.

For receive operations, the implementation of the operation will set the message on the aio on success, and the consuming application hasa responsibility to retrieve and dispose of the message. Failure to do so will leak the message. If the operation does not complete successfully, then no message is stored on the aio.

I/O Vector

typedef struct nng_iov {
    void * iov_buf;
    size_t iov_len;
};

int nng_aio_set_iov(nng_aio *aio, unsigned int niov, nng_iov *iov);

For some operations, the unit of data transferred is not a message, but rather a stream of bytes.

For these operations, an array of niov nng_iov structures can be passed to the nng_aio_set_iov function to provide a scatter/gather array of elements describing the location (iov_buf) and length (iov_len) of data, to transfer.

The iov vector is copied into storage in the aio itself, so that callers may use stack allocated nng_iov structures. The values pointed to by the iov_buf members are not copied by this function though.

A maximum of four (4) nng_iov members may be supplied.

Inputs and Outputs

void nng_aio_set_input(nng_aio *aio, unsigned int index, void *param);
void *nng_aio_get_output(nng_aio *aio, unsigned int index);

Asynchronous operations can take additional input parameters, and provide additional result outputs besides the result code.

The nng_aio_set_input function sets the input parameter at index to param for the operation associated with aio.

The nng_aio_get_output function returns the output result at index for the operation associated with aio.

The type and semantics of input parameters and output results are determined by specific operations. The documentation for the operation should provide details.

The valid values of index range from zero (0) to three (3), as no operation currently defined can accept more than four parameters or return more than four additional results.

Scatter Gather Vectors

typedef struct nng_iov {
	void  *iov_buf;
	size_t iov_len;
} nng_iov;

void nng_aio_set_iov(nng_aio *aio, unsigned nio, const nng_iov *iov);

Some asynchronous operations, such as those dealing with streams, use scatter or gather vectors, where data to be transferred is either gathered from multiple separate regions of memory, or scattered into separate regions of memory. For example a message may have a header located at one location in memory, and a payload located in another.

The nng_aio_set_iov function configures the aio to use nio separate segments, described by the elements in iov. For each of these, the segment of size iov_len located at iov_buf will be used.

The elements of iov will be copied into aio, so the vector may be located on the stack or another temporary location. The locations referenced by it, must remain valid for the duration of the operation, of course.

Note that many of these operations are not guaranteed to perform a full transfer of data, so even a successful operation should check the amount of data actually transferred using nng_aio_count, and if necessary resubmit the operation with a suitably updated vector of nng_iov using this function.

See Also

Synchronization, Threads, Streams, Time

Synchronization Primitives

In order to allow safely accessing shared state, or to allow coordination between different threads, NNG provides synchronization primitives in the form of mutual exclusion locks and condition variables.

Correct use of these primitives will be needed when accessing shared state from threads, or from callback functions associated with asynchronous operations. (The need to do this in callbacks is because the callback may be executed under a task thread other than the submitting thread.)

Mutual Exclusion Lock

typedef struct nng_mtx nng_mtx;

Mutual exclusion locks, or mutex locks, represented by the nng_mtx structure, allow only a single thread to lock “own” the lock, acquired by nng_mtx_lock. Any other thread trying to acquire the same mutex will wait until the owner has released the mutex by calling nng_mtx_unlock.

Creating a Mutex

int nng_mutx_alloc(nng_mt **mtxp);

A mutex can be created by allocating one with nng_mtx_lock. On success, a pointer to the mutex is returned through mtxp. This function can fail due to insufficient memory or resources, in which case it will return NNG_ENOMEM. Otherwise it will succceed and return zero.

Destroying a Mutex

void nng_mtx_free(nng_mtx *mtx);

When no longer needed, a mutex can be deallocated and its resources returned to the caller, by calling nng_mtx_free. The mutex must not be locked by any thread when calling this function.

Acquiring a Mutex

void nng_mtx_lock(nng_mtx *mtx);

The nng_mtx_lock function acquires ownership of a mutex, waiting for it to unowned by any other threads if necessary.

Releasing a Mutex

void nng_mtx_unlock(nng_mtx *mtx);

The nng_mtx_unlock function releases a mutex that the calling thread has previously acquired with nng_mtx_lock.

Condition Variable

typedef struct nng_cv nng_cv;

The nng_cv structure implements a condition variable, associated with the the mutex mtx which was supplied when it was created.

Condition variables provide for a way to wait on an arbitrary condition, and to be woken when the condition is signaled. The mutex is dropped while the caller is asleep, and reacquired atomically when the caller is woken.

Creating a Condition Variable

int nng_cv_alloc(nng_cv **cvp, nng_mtx *mtx);

The nng_cv_alloc function allocates a condition variable, and associated with the mutex mtx, and returns a pointer to it in cvp.

Destroy a Condition Variable

void nng_cv_free(nng_cv *cv);

The nng_cv_free function deallocates the condition variable cv.

Waiting for the Condition

int nng_cv_until(nng_cv *cv, nng_time when);
void nng_cv_wait(nng_cv *cv);

The nng_cv_until and nng_cv_wait functions put the caller to sleep until the condition variable cv is signaled, or (in the case of nng_cv_until), the specified time when (as determined by nng_clock is reached.

While nng_cv_wait never fails and so has no return value, the nng_cv_until function can return NNG_ETIMEDOUT if the time is reached before condition cv is signaled by either nng_cv_wake or nng_cv_wake1.

Signaling the Condition

void nng_cv_wake(nng_cv *cv);
void nng_cv_wake1(nng_cv *cv);

The nng_cv_wake and nng_cv_wake1 functions wake threads waiting in nng_cv_until or nng_cv_wait. The difference between these functions is that nng_cv_wake will wake every thread, whereas nng_cv_wake1 will wake up exactly one thread (which may be chosen randomly).

Examples

Example 1: Allocating the condition variable

nng_mtx *m;
nng_cv *cv;
nng_mtx_alloc(&m); // error checks elided
nng_cv_alloc(&cv, m);

Example 2: Waiting for the condition

expire = nng_clock() + 1000; // 1 second in the future
nng_mtx_lock(m);  // assume cv was allocated using m
while (!condition_true) {
  if (nng_cv_until(cv, expire) == NNG_ETIMEDOUT) {
    printf("Time out reached!\n");
    break;
  }
}
// condition_true is true
nng_mtx_unlock(m);

Example 3: Signaling the condition

nng_mtx_lock(m);
condition_true = true;
nng_cv_wake(cv);
nng_mtx_unlock(m);

See Also

Threads, Time, Asynchronous I/O

Threads

Threads provide a means of representing multiple parallel execution contexts. NNG makes use of this concept internally, but also provides for user applications to utilize the same thread facilities. This allows one form of concurrency for applications.

Thread Structure

typedef struct nng_thread nng_thread;

The nng_thread structure represnts a thread, which is a single execution context. A given thread will have its own stack, and CPU registers. However global state, as well as values allocated on the heap, will be shared and accessible to all threads in the system (See the Synchronization chapter for functions to help with data sharing between different threads.)

Multiple threads can be thought of as running concurrently, even though they might not actually do so.

I/O operations that block (i.e. wait for completion) will block the thread, while allowing other threads to proceed.

Creating a Thread

int nng_thread_create(nng_thread **thrp, void (*func)(void *), void *arg);

The nng_thread_create function creates a thread, which will execute func, with the given argument arg, and returns a pointer to it in thrp.

The thread may begin execution immediately.

The thread will persist until func returns.

This function returns zero on success, but may return NNG_ENOMEM if insufficient resources to create a thread are available.

Destroying a Thread

void nng_thread_destroy(nng_thread *thr);

The nng_thread_destroy function waits for the thread thr to finish execution. This function should be called to reclaim any resources associated with the thread when done. It also has the effect of blocking execution in the caller until thr has completed executing.

Thread Names

void nng_thread_set_name(nng_thread *thr, const char *name);

In order to facilitate debugging, nng_thread_set_name may be called to provide a name for the thread. This may change how the thread is represented in debuggers. Not all platforms support setting the thread name.

See Also

Synchronization, Asynchronous Operations

Logging

This chapter describes the support for message logs. Both applications and NNG itself can emit logs, which can be useful for application field support and debugging. Additionally applications can customize the handling of this logging as needed.

Note that logging is disabled by default unless an application configures a suitable logger with nng_log_set_logger.

Submitting Logs

void nng_log_err(const char *msgid, const char *msg, ...);
void nng_log_warn(const char *msgid, const char *msg, ...);
void nng_log_notice(const char *msgid, const char *msg, ...);
void nng_log_info(const char *msgid, const char *msg, ...);
void nng_log_debug(const char *msgid, const char *msg, ...);

These functions inject a a message into the logging system, where it will be processed and potentially go to system logs, standard output, or procssed further.

The msgid is a short prefix that should uniquely identify the message, possibly also with some kind of category. It is recommended that strings between 8 and 16 charactes be used. As this may, but will not necessarily be displayed to the user, the content of the message should not appear solely in this field. A NULL value is permitted here, but that may make filtering the message or other automatic processing more difficult.

The msg is a printf-style format string, which is used to format the message content. The following arguments are consumed in the same manner as printf.

Auth Logs

void nng_log_auth(nng_log_level level, const char *msgid, const char *msg, ...);

The nng_log_auth function formats and injects a security related log message. (“Auth” can indicate either “authentication” or “authorization”.) The level is a log level. The msgid, msg, and any remaining arguments are processed in a fashion similar to the other logging functions, except that the logs may be are logged using the NNG_LOG_AUTH facility, and thus may be redirected or receive other special treatment.

Log Levels

typedef enum nng_log_level nng_log_level;

void nng_log_set_level(nng_log_level level);
nng_log_level nng_log_get_level(void);

The nng_log_level type represents a severity for logged messages. These levels correspond to those found in the UNIX syslog subsystem, although applications should not depend upon the values being identical.

The nng_log_set_level function sets the log level. Messages with a severity that is numerically greater than this (less-severe) will be discarded.

The nng_log_get_level function returns the log level most recently set by nng_log_set_level or the default if that function has not been called.

The log levels are defined as follows:

typedef enum nng_log_level {
    NNG_LOG_NONE   = 0, // used for filters only, NNG suppresses these
    NNG_LOG_ERR    = 3,
    NNG_LOG_WARN   = 4,
    NNG_LOG_NOTICE = 5,
    NNG_LOG_INFO   = 6,
    NNG_LOG_DEBUG  = 7
} nng_log_level;

The value NNG_LOG_NONE may be useful to suppress message logs altogether.

The default level is typically NNG_LOG_NOTICE, but applications should select a value rather than relying upon the default.

Log Facilities

typedef enum nng_log_facility

void nng_log_set_facility(nng_log_facility facility);

Logging facilities are used to indicate the source of a log message, and may be useful in routing and processing these logs. Traditionally these are used with the UNIX syslog system, and the values here represent some (but not all) of the values found there.

The following values are defined:

typedef enum nng_log_facility {
    NNG_LOG_USER   = 1,
    NNG_LOG_DAEMON = 3,
    NNG_LOG_AUTH   = 10,
    NNG_LOG_LOCAL0 = 16,
    NNG_LOG_LOCAL1 = 17,
    NNG_LOG_LOCAL2 = 18,
    NNG_LOG_LOCAL3 = 19,
    NNG_LOG_LOCAL4 = 20,
    NNG_LOG_LOCAL5 = 21,
    NNG_LOG_LOCAL6 = 22,
    NNG_LOG_LOCAL7 = 23,
} nng_log_facility;

The nng_log_set_facility function can be used to set the facility that the application will use when emitting log messages. This should be called as part of initialization of the application, if logging is to be used.

The default facility is typically NNG_LOG_USER, but applications should select a value rather than relying upon the default.

Log Handlers

typedef void (*nng_logger)(nng_log_level level, nng_log_facility facility,
    const char *msgid, const char *msg);

void nng_null_logger(nng_log_level, nng_log_facility, const char *, const char *);
void nng_stderr_logger(nng_log_level, nng_log_facility, const char *, const char *);
void nng_system_logger(nng_log_level, nng_log_facility, const char *, const char *);

void nng_log_set_logger(nng_logger logger);

Log handlers are responsible for actually processing the logged messages.

The nng_log_set_logger function installs the named logger, of type nng_logger, as the log handler. The function logger will be called when any message is meant to be processed. (Messages are first filtered by severity, then formatted, before calling the logger.)

Any previously installed logger is replaced by logger.

The nng_null_logger function is an implementation of nng_logger that simply discards the content. This is the default logger, so logging is disabled by default.

The nng_stderr_logger function is an implementation that logs messages to the standard error stream. It will attempt to colorize messages by the severity, if the standard error is a terminal device. This can be suppressed by setting either the NO_COLOR or NNG_LOG_NO_COLOR environment variables.

The nng_system_logger attempts to use an appropriate system facility to log messages. For POSIX systems, this means using syslog to process the messages. For other systems the defauilt behavior may be the same as nng_stderr_logger.

See Also

The Syslog Protocol upon which this is based is documented in the following two IETF RFCS,

• R. Gerhards, RFC 5424, The Syslog Protocol, March 2009
• C. Lonvick, RFC 3164, The BSD syslog Protocol, August 2001

Statistics

To facilitate debugging and support situations, the NNG library provides for collection and reporting of numerous statistics.

These statistics are organized in a tree, and include both values, and metadata describing the statistics. In order to be efficient and minimize the impact of maintaining statistics, an explicit snapshot of statistics must be taken, and that snapshot can then be processed.

Statistic Structure

typedef struct nng_stat nng_stat;

The nng_stat structure represents a statistic, which is a single value collected at a specific point in time.

This structure has meta-data describing the value, the value itself, and links to any sibling or child statistics.

Collecting a Snapshot

int nng_stats_get(nng_stat **statsp);

The nng_stats_get function takes a snapshot of the statistics for the system and returns it through the pointer statsp. This function may return NNG_ENOMEM if memory is exhausted, or NNG_ENOTSUP if the statistics support is not enabled in the build, but is otherwise expected to return zero.

Freeing a Snapshot

void nng_stats_free(nng_stat *stat);

The nng_stats_free function deallocates the snapshot referenced by stat.

Traversing the Tree

const nng_stat *nng_stat_child(const nng_stat *stat);
const nng_stat *nng_stat_next(const nng_stat *stat);
const nng_stat *nng_stat_parent(const nng_stat *stat);

Traversing the tree of statistics is done using the nng_stat_child and nng_stat_next functions.

The nng_stat_child function returns either the first child of stat, or NULL if the stat has no children.

The nng_stat_next function returns the nearest sibling to the right of stat, or NULL if stat has no more siblings to the right.

The nng_stat_parent function returns the parent of stat, or NULL if stat is the root of the statistics tree.

Finding a Statistic

const nng_stat *nng_stat_find(const nng_stat *stat, const char *name);
const nng_stat *nng_stat_find_dialer(const nng_stat *stat, nng_dialer dialer);
const nng_stat *nng_stat_find_listener(const nng_stat *stat, nng_dialer listener);
const nng_stat *nng_stat_find_socket(const nng_stat *stat, nng_dialer socket);

Sometimes it is easiest to search for a specific statistic, matching by name, or possibly to find the tree of statistics associated iwth a specific socket, dialer, or listener.

The nng_stat_find functions are provided for this purpose.

The nng_stat_find function returns the first statistic within the subtree of statistics stat, with the given name. If no such statistic can be found, NULL is returned.

The nng_stat_find_dialer, nng_stat_find_listener, and nng_stat_find_socket return the statistics subtree for the given dialer, listener, or socket object. If no such statistic can be found, then they return NULL. These functions should be provided the root of the statistic tree, in order to ensure that they can find the desired object.

Statistic Identification

const char *nng_stat_name(const nng_stat *stat);
const char *nng_stat_desc(const nng_stat *stat);

Every statistic has a name, returned by nng_stat_name, and a description, returned by nng_stat_desc. Descriptions are human-readable text, which might be useful for display.

Statistic Type

int nng_stat_type(const nng_stat *stat);

The function nng_stat_type returns the type of the statistic. The type of a statistic determines the nature of the value, and which function can be used to obtain that value.

• NNG_STAT_SCOPE: The statistic does not carry any real value, but is used for grouping related statistics together. This is a nexus in the statistics tree.

• NNG_STAT_COUNTER: The statistic is a counter that only increments. Usually the change in the value of the statistic is more interesting (as a rate) than the absolute value at any given time. The value should be obtained using nng_stat_value. The units will be given by the value returned from nng_stat_unit.

• NNG_STAT_LEVEL: The statistic represnts a measured value which corresponds to a specific value at a specific time. For example, this may represent the number of messages currently queued for some operation, or the link speed of a network interface. Most often the absolute value is more interesting than the change in the value over time. Again the value can be obtained with nng_stat_value, and any appropriate unit of measurement with nng_stat_unit.

• NNG_STAT_STRING: The statistic is a string, such as a name. The value of the string can be obtained with nng_stat_string. The value of this string will remain valid until the snapshot is deallocated with nng_stats_free.

• NNG_STAT_BOOLEAN: The value of the statistic is a truth value (either true or false) and can be obtained with nng_stat_bool.

• NNG_STAT_ID: The value of the statistic is a numeric identifier, such as a socket identifier. The value can be obtained with nng_stat_value, and will be fixed for the life of the statistic.

Statistic Value

uint64_t nng_stat_value(const nng_stat *stat);
const char *nng_stat_string(const nng_stat *stat);
bool nng_stat_bool(const nng_stat *stat);

These functions return the value associated with the statistic.

The nng_stat_value function returns the the numeric value for the statistic stat of type NNG_STAT_COUNTER, NNG_STAT_LEVEL, or NNG_STAT_ID. If stat is not one of these types, then it returns zero.

The nng_stat_bool function returns the Boolean value (either true or false) for the statistic stat of type NNG_STAT_BOOLEAN. If the statistics is not of this type, then it returns false.

The nng_stat_string function returns a pointer to a string value for the statistic stat, of type NNG_STAT_STRING. This string will remain valud until the snapshot that stat was collected with is deallocated with nng_stats_free. If the statistic is not of type NNG_STAT_STRING, then NULL is returned.

Statistic Units

int nng_stat_unit(const nng_stat *stat);

For statistics of type NNG_STAT_COUNTER or NNG_STAT_LEVEL, it is often useful to know what that quantity being reported measures. The following units may be returned from nng_stat_unit for such a statistic:

• NNG_UNIT_NONE: No unit is known or applies.
• NNG_UNIT_BYTES: A count of bytes.
• NNG_UNIT_MESSAGES: A count of messages.
• NNG_UNIT_MILLIS: A count of milliseconds.
• NNG_UNIT_EVENTS: A count of events of some type.

Statistic Timestamp

uint64_t nng_stat_timestamp(const nng_stat *stat);

Statistics have a timestamp indicating when the value was sampled, obtained via nng_stat_timestamp. The timestamp is given in in milliseconds since a reference time, and the reference time used here is the same reference time used for nng_clock.

See Also

nng_clock

Errors

typedef enum ... nng_err;

Many NNG functions can fail for a variety of reasons. These functions tend to return either zero on success, or a non-zero error code to indicate failure. ¹

All these error codes are nng_err.

Not every possible error code is defined here, as sometimes an underlying system or library error code is “wrapped”.

Human Readable Error Message

const char *nng_strerror(nng_err err);

The nng_strerror returns the human-readable description of the given error in err.

The error message returned is a fixed NUL-terminated string and may be located in read-only memory.

The returned error message is provided in US English, but in the future locale-specific strings may be presented instead.

List of Errors

1: This convention goes back to UNIX system calls, which behave the same way, but NNG does not use a separate errno variable.

Streams

NNG provides a common streams API for working with byte-oriented streams. In NNG, streams are bidirectional connections for exchanging a stream of bytes.

Some common examples of streams are TCP connections, UNIX domain sockets, Windows named pipes, and WebSockets.

The API documented here is to facilitate applications that wish to work with these at a lower-level than Scalability Protocols, in a way that is both portable and agnostic about the specific underlying transport mechanism.

Stream Type

typedef struct nng_stream nng_stream

The base nng_stream type represents a bidirectional, byte-oriented, reliable connection.

Sending and Receiving Data

void nng_stream_send(nng_stream *s, nng_aio *aio);
void nng_stream_recv(nng_stream *s, nng_aio *aio);

The nng_stream_send function starts sending data asynchronously over the stream s. The data is sent from the scatter/gather vector located in the nng_aio aio, which must have been previously set using nng_aio_set_iov.

The nng_stream_recv function starts receiving data [asynchronously over the stream s into the scatter/gather vector located in the nng_aio aio, which must have been previously set using nng_aio_set_iov.

These functions return immediately, with no return value. Completion of the operation is signaled via the aio, and the final result may be obtained via nng_aio_result.

The I/O operation may complete as soon as at least one byte has been transferred, or an error has occurred. Therefore, the number of bytes transferred may be less than requested. The actual number of bytes transferred can be determined with nng_aio_count.

Closing a Stream

void nng_stream_close(nng_stream *s);
void nng_stream_stop(nng_stream *s);
void nng_stream_free(nng_stream *s);

The nng_stream_close function closes a stream, but does not destroy it. This function returns immediately. Operations that are pending against the stream, such as nng_stream_send or nng_stream_recv operations will be canceled asynchronously, if possible. Those operations will result in NNG_ECLOSED.

The nng_stream_stop function not only closes the stream, but waits for any operations pending against it to complete, and for any underlying asynchronous registered I/O to be fully deregistered. As some systems use separate threads for asynchronous I/O, stopping the stream is necessary before those resources can be freed. Until the stream is stopped, there could still be I/O operations in flight, making it unsafe to deallocate memory.

The nng_stream_free function stops the stream like nng_stream_stop, but also deallocates the stream itself.

Getting Stream Options

int nng_stream_get_bool(nng_stream *s, const char *opt, bool *valp);
int nng_stream_get_int(nng_stream *s, const char *opt, int *valp);
int nng_stream_get_ms(nng_stream *s, const char *opt, nng_duration *valp);
int nng_stream_get_size(nng_stream *s, const char *opt, size_t *valp);
int nng_stream_get_addr(nng_stream *s, const char *opt, nng_sockaddr *valp);
int nng_stream_get_string(nng_stream *s, const char *opt, char **valp);

These functions are used to obtain value of an option named opt from the stream s, and store it in the location referenced by valp.

These functions access an option as a specific type. The transport layer will have details about which options are available, and which type they may be accessed using.

In the case of nng_stream_get_string, the string is created as if by nng_strdup, and must be freed by the caller using nng_strfree when no longer needed.

Stream Factories

typedef struct nng_stream_dialer nng_stream_dialer;
typedef struct nng_stream_listener nng_stream_listener;

The nng_stream_listener object and nng_stream_listener objects can be thought of as factories that create nng_stream streams.

The nng_stream_listener object a handle to a listener, which creates streams by accepting incoming connection requests. In a BSD socket implementation, this is the entity responsible for doing bind, listen and accept. Normally a listener may be used to accept multiple, possibly many, concurrent connections.

The nng_stream_dialer object is a handle to a dialer, which creates streams by making outgoing connection requests. While there isn’t a specific BSD socket analogue, this can be thought of as a factory for TCP sockets created by opening them with socket and then calling connect on them.

Creating a Stream Factory

int nng_stream_dialer_alloc(nng_stream_dialer **dialerp, const char *url);
int nng_stream_dialer_alloc_url(nng_stream_dialer **dialerp, const nng_url *url);
int nng_stream_listener_alloc(nng_stream_listener **lstenerp, const char *url);
int nng_stream_listener_alloc_url(nng_stream_listener **listenerp, const nng_url *url);

The nng_stream_dialer_alloc and nng_stream_dialer_alloc_url functions create a stream dialer, associated the URL specified by url represented as a string, or as an nng_url object, respectively. The dialer is returned in the location dialerp references.

The nng_stream_listener_alloc and nng_stream_listener_alloc_url functions create a stream listener, associated the URL specified by url represented as a string, or as an nng_url object, respectively. The listener is returned in the location listenerp references.

Example 1: Creating a TCP Listener

This shows creating a TCP listener that listens on INADDR_ANY, port 444.

nng_listener listener;
int rv = nng_stream_listener_alloc(&listener, "tcp://:444");

Closing a Stream Factory

void nng_stream_dialer_close(nng_stream_listener *dialer);
void nng_stream_dialer_stop(nng_stream_listener *dialer);
void nng_stream_dialer_free(nng_stream_listener *dialer);
void nng_stream_listener_close(nng_stream_listener *listener);
void nng_stream_listener_stop(nng_stream_listener *listener);
void nng_stream_listener_free(nng_stream_listener *listener);

The nng_stream_dialer_close and nng_stream_listener_close functions close the stream dialer or listener, preventing it from creating new connections. This will generally include closing any underlying file used for creating such connections. However, some requests may still be pending when this function returns, as it does not wait for the shutdown to complete.

The nng_stream_dialer_stop and nng_stream_listener_stop functions performs the same action, but also wait until all outstanding requests are serviced, and the dialer or listener is completely stopped. Because they blocks, these functions must not be called in contexts where blocking is not allowed.

The nng_stream_dialer_free and nng_stream_listener_free function performs the same action as nng_stream_dialer_stop or nng_stream_listener_stop, but also deallocates the dialer or listener, and any associated resources.

Making Outgoing Connections

void nng_stream_dialer_dial(nng_stream_dialer *dialer, nng_aio *aio);

The nng_stream_dialer_dial initiates an outgoing connection asynchronously, using the nng_aio aio. If it successfully establishes a connection, it creates an nng_stream, which can be obtained as the first output result on aio using the nng_aio_get_output function with index zero.

Example 3: Connecting to Google

This demonstrates making an outbound connection to “google.com” on TCP port 80. Error handling is elided for clarity.

nng_aio *aio;
nng_stream_dialer *dialer;
nng_stream *stream;

nng_stream_dialer_alloc(&dialer, "tcp://google.com:80");

nng_aio_alloc(&aio, NULL, NULL);

// make a single outbound connection
nng_stream_dialer_dial(dialer, aio);
nng_aio_wait(aio); // wait for the asynch operation to complete
if (nng_aio_result(aio) != 0) {
    // ... handle the error
}
stream = nng_aio_get_output(aio, 0);

Accepting Incoming Connections

int nng_stream_listener_listen(nng_stream_listener *listener);
void nng_stream_listener_accept(nng_stream_listener *listener, nng_aio *aio);

Accepting incoming connections is performed in two steps. The first step, nng_stream_listener_listen is to setup for listening. For a TCP implementation of this, for example, this would perform the bind and the listen steps. This will bind to the address represented by the URL that was specific when the listener was created with nng_stream_listener_alloc.

In the second step, nng_stream_listener_accept accepts an incoming connection on listener asynchronously, using the nng_aio aio. If an incoming connection is accepted, it will be represented as an nng_stream, which can be obtained from the aio as the first output result using the nng_aio_get_output function with index zero.

Example 3: Accepting an Inbound Stream

For clarity this example uses a synchronous approach using nng_aio_wait, but a typical server application would most likely use a callback to accept the incoming stream, and start another instance of nng_stream_listener_accept.

nng_aio *aio;
nng_listener *listener;
nng_stream *stream;

nng_stream_listener_alloc(&listener, "tcp://:8181");
nng_aio_alloc(&aio, NULL, NULL); // normally would use a callback

// listen (binding to the URL in the process)
if (nng_stream_listener_listen(listener)) {
    // ... handle the error
}

// now accept a single incoming connection as a stream object
nng_stream_listener_accept(l, aio);
nng_aio_wait(aio); // wait for the asynch operation to complete
if (nng_aio_result(aio) != 0) {
    // ... handle the error
}
stream = nng_aio_get_output(aio, 0);

Stream Factory Options

int nng_stream_dialer_get_addr(nng_stream_dialer *dialer, const char *opt, nng_sockaddr *valp);
int nng_stream_dialer_get_bool(nng_stream_dialer *dialer, const char *opt, bool *valp);
int nng_stream_dialer_get_int(nng_stream_dialer *dialer, const char *opt, int *valp);
int nng_stream_dialer_get_ms(nng_stream_dialer *dialer, const char *opt, nng_duration *valp);
int nng_stream_dialer_get_size(nng_stream_dialer *dialer, const char *opt, size_t *valp);
int nng_stream_dialer_get_string(nng_stream_dialer *dialer, const char *opt, char **valp);

int nng_stream_listener_get_addr(nng_stream_listener *listener, const char *opt, nng_sockaddr *valp);
int nng_stream_listener_get_bool(nng_stream_listener *listener, const char *opt, bool *valp);
int nng_stream_listener_get_int(nng_stream_listener *listener, const char *opt, int *valp);
int nng_stream_listener_get_ms(nng_stream_listener *listener, const char *opt, nng_duration *valp);
int nng_stream_listener_get_size(nng_stream_listener *listener, const char *opt, size_t *valp);
int nng_stream_listener_get_string(nng_stream_listener *listener, const char *opt, char **valp);

int nng_stream_dialer_set_addr(nng_stream_dialer *dialer, const char *opt, const nng_sockaddr *val);
int nng_stream_dialer_set_bool(nng_stream_dialer *dialer, const char *opt, bool val);
int nng_stream_dialer_set_int(nng_stream_dialer *dialer, const char *opt, int val);
int nng_stream_dialer_set_ms(nng_stream_dialer *dialer, const char *opt, nng_duration val);
int nng_stream_dialer_set_size(nng_stream_dialer *dialer, const char *opt, size_t val);
int nng_stream_dialer_set_string(nng_stream_dialer *dialer, const char *opt, const char *val);

int nng_stream_listener_set_addr(nng_stream_listener *listener, const char *opt, const nng_sockaddr *val);
int nng_stream_listener_set_bool(nng_stream_listener *listener, const char *opt, bool val);
int nng_stream_listener_set_int(nng_stream_listener *listener, const char *opt, int val);
int nng_stream_listener_set_ms(nng_stream_listener *listener, const char *opt, nng_duration val);
int nng_stream_listener_set_size(nng_stream_listener *listener, const char *opt, size_t val);
int nng_stream_listener_set_string(nng_stream_listener *listener, const char *opt, const char *val);

These functions are used to retrieve or change the value of an option named opt from the stream dialer or listener. The nng_stream_dialer_get_ and nng_stream_listener_get_ function families retrieve the value, and store it in the location valp references. The nng_stream_dialer_set_ and nng_stream_listener_set_ function families change the value for the dialer or listener, taking it from val.

These functions access an option as a specific type. The transport layer will have details about which options are available, and which type they may be accessed using.

In the case of nng_stream_dialer_get_string and nng_stream_listener_get_string, the string is created as if by nng_strdup, and must be freed by the caller using nng_strfree when no longer needed.

In the case of nng_stream_dialer_set_string and nng_stream_listener_set_string, the string contents are copied if necessary, so that the caller need not retain the value referenced once the function returns.

In the case of nng_stream_dialer_set_addr and nng_stream_listener_set_addr, the contents of addr are copied if necessary, so that the caller need not retain the value referenced once the function returns.

Example 4: Socket Activation

Some nng_stream_listener objects, depending on the underlying transport and platform, can support a technique known as “socket activation”, where the file descriptor used for listening and accepting is supplied externally, such as by a system service manager. In this case, the application supplies the file descriptor or SOCKET object using the NNG_OPT_LISTEN_FD option, instead of calling nng_stream_listener_listen.

nng_stream_listener *listener;
int fd;

// This is a systemd API, not part of NNG.
// See systemd documentation for an explanation.
// fd at this point has already had bind() and listen() called.
fd = SD_LISTEN_FDS_START + 0;

nng_stream_listener_alloc(&listener, "tcp://");
nng_stream_listener_set_int(listener, NNG_OPT_LISTEN_FD, fd);

// can now start doing nng_stream_listener_accept...

TLS Configuration

int nng_stream_dialer_get_tls(nng_stream_listener *dialer, nng_tls_config **tlsp);
int nng_stream_dialer_set_tls(nng_stream_listener *dialer, nng_tls_config *tls);
int nng_stream_listener_get_tls(nng_stream_listener *listener, nng_tls_config **tlsp);
int nng_stream_listener_set_tls(nng_stream_listener *listener, nng_tls_config *tls);

Both nng_stream_dialer and nng_stream_listener objects may support configuration of TLS parameters. The nng_stream_dialer_set_tls and nng_stream_listener_set_tls functions support setting the configuration of a nng_tls_config object supplied by tls on dialer or listener. This must be performed before the listener starts listening with nng_stream_listener_listen, or the dialer starts an outgoing connection as a result of nng_stream_dialer_dial.

The configuration object that was previously established (which may be a default if one was not explicitly configured) can be obtained with the nng_stream_dialer_get_tls and nng_stream_listener_get_tls. They will return a pointer to the nng_tls_config object in question at the location referenced by tlsp.

HTTP Support

NNG offers support for creation of HTTP clients, and servers. NNG supports HTTP/1.1 at present, and supports a subset of functionality, but the support should be sufficient for simple clients, REST API servers, static content servers, and gateways between HTTP and and other protocols. It also provides support for WebSocket based connections.

HTTP follows a request/reply model, where a client issues a request, and the server is expected to reply. Every request is answered with a single reply.

Header File

#include <nng/http.h>

Unlike the rest of NNG, the HTTP API in NNG requires including nng/http.h. It is not necessary to include the main nng/nng.h header, it will be included transitively by nng/http.h.

Connection Object

typedef struct nng_http nng_http;

The nng_http object represents a single logical HTTP connection to the server. For HTTP/1.1 and earlier, this will correspond to a single TCP connection, but the object also contains state relating to the transaction, such as the hostname used, HTTP method used, request headers, response status, response headers, and so forth.

An nng_http object can be reused, unless closed, so that additional transactions can be performed after the first transaction is complete.

At any given point in time, an nng_http object can only refer to a single HTTP transaction. In NNG, these nng_http objects are used in both the client and server APIs.

The nng_http object is created by either nng_http_client_connect or by an HTTP server object which then passes it to an [nng_http_handler] callback function.

HTTP Method

void nng_http_set_method(nng_http *conn, const char *method);
const char *nng_http_get_method(nng_http *conn);

Each HTTP transaction has a single verb, or method, that is used. The most common methods are “GET”, “HEAD”, and “POST”, but a number of others are possible.

The nng_http_set_method function specifies the HTTP method to use for the transaction. The default is “GET”. HTTP methods are case sensitive, and generally upper-case, such as “GET”, “POST”, “HEAD”, and so forth. This function silently truncates any method to 32-characters. (There are no defined methods longer than this.)

The nng_http_get_method function is used, typically on a server, to retrieve the method the client set when issuing the transaction.

HTTP URI

nng_err nng_http_set_uri(nng_http *conn, const char *uri, const char *query);
const char *nng_http_get_uri(nng_http *conn);

The nng_http_set_uri function sets the URI, which normally appears like a path such as “/docs/index.html”, for the next transaction on conn. It sets the URI to uri, and, if query is not NULL, also appends the contents of query, separated by either the ‘?’ or ‘&’ character, depending on whether uri already contains a query string. It may return NNG_ENOMEM, or NNG_EMSGSIZE if the the result is too long, or NNG_EINVAL if there is some other problem with the URI.

The nni_http_get_uri function is used to obtain the URI that was previously set by nng_http_set_uri. If the URI is unset (such as for a freshly created connection), then it returns NULL. The returned value will have any query concentated, for example “/api/get_user.cgi?name=garrett”.

HTTP Version

nng_err nng_http_set_version(nng_http *conn, const char *version);
const char *nng_http_get_version(nng_http *conn);

The nng_http_set_version function is used to select the HTTP protocol version to use for the exchange. At present, only the values NNG_HTTP_VERSION_1_0 and NNG_HTTP_VERSION_1_1 (corresponding to “HTTP/1.0” and “HTTP/1.1”) are supported. NNG will default to using “HTTP/1.1” if this function is not called. If an unsupported version is supplied, NNG_ENOTSUP will be returned, otherwise zero.

The nng_http_get_version function is used to determine the version the client selected. Normally there is little need to use this, but there are some subtle semantic differences between HTTP/1.0 and HTTP/1.1.

HTTP Status

typedef enum ... nng_http_status;
nng_http_status nng_http_get_status(nng_http *conn);
const char *nng_http_get_reason(nng_http_conn *conn);
void nng_http_set_status(nng_http *conn, nng_http_status status, const char *reason);

The nng_http_get_status function obtains the numeric code (typipcally numbered from 100 through 599) returned by the server in the last exchange on conn. (If no exchange has been performed yet, the result is undefined.) The value is returned as an nng_http_status.

A descriptive message matching the status code is returned by nng_http_get_reason.

The nng_http_set_status function is used on a server in a handler callback to set the status code that will be reported to the client to status, and the associated text (reason) to reason. If reason is NULL, then a built in reason based on the status will be used instead.

Status codes are defined by the IETF. Here are defininitions that NNG provides for convenience:

Retrieving Headers

const char *nng_http_get_header(nng_http *conn, const char *key);
bool nng_next_header(nng_http *conn, const char **keyp, const char **valuep, void **next);

The nng_http_get_header returns the header value matching key that was received over conn, or NULL if no such header exists.

Thus, if conn is a client connection, then this function returns the the header value sent by the server as part of a response, whereas if it is a server connection, it returns the header value sent by the client as part of the request.

If multiple headers are present with the same key, they may be returned as a combined value, with individual values separated by commas, but this behavior is not guaranteed.

The nng_http_next_header function iterates over all the headers, using the same list that nng_http_get_header uses. To start, it is called with next initialized to NULL. If a header was found, then it returns true, and sets keyp and valuep to values containing the header name and value. It also updates next, which should be used for the next iteration.

Once nng_http_next_header returns false, further calls with the same parameters will continue to do so. The scan can be rest by setting next to NULL.

Modifying Headers

nng_err nng_http_add_header(nng_http *conn, const char *key, const char *val);
nng_err nng_http_set_header(nng_http *conn, const char *key, const char *val);
void nng_http_del_header(nng_http *conn, const char *key);

The nng_http_add_header, nng_http_set_header, and nng_http_del_header functions are used to add a modify either the request or response headers for conn prior to sending to the connected peer on conn.

Thus, if the conn is a client connection created by nng_http_client_connect, then the request headers are modified. Conversely, if it is a connection created by an HTTP server and used in a callback function, then the response headers are modified.

The nng_http_add_header function adds a header with the name key, and the value val, to the list of headers. In so doing, it may bring collapse multiple headers with the same name into a comma separated list, following the syntax specified in RFC 9110. The function may return NNG_ENOMEM, NNG_EMSGSIZE, or NNG_EINVAL.

The nng_http_set_header function adds the header if it does not already exist, but replaces any and all previously existing headers with the same name key, if they exist. In all other respects it behaves similarly to nng_http_add_header.

The nng_http_del_header removes all headers with name key.

Retrieving Body Content

void nng_http_get_body(nng_http_conn *conn, void **datap, size_t *sizep);

The nng_http_get_data obtains the most recently received request or response body. This will be NULL if the content has not been retrieved properly yet, or if the peer did not any content. (Some requests are defined to never have body content, such as “HEAD”.)

Storing Body Content

void nng_http_set_body(nng_http_conn *conn, void *data, size_t size);
void nng_http_copy_body(nng_http_conn *conn, const void *data, size_t size);

The nng_http_set_data function sets the outgoing body content to data, which must be size bytes long. The caller must ensure that data remains valid for the duration of the transaction.

The nng_http_copy_data function makes a copy of data, which will be freed automatically when the transaction is finished, but otherwise behaves like nng_http_set_data.

On client conn objects, these functions update the request object, but on server conn objects, they update the response object.

These functions also update the relevant “Content-Length” header.

Closing the Connection

void nng_http_close(nng_http *conn);

The nng_http_close function closes the supplied HTTP connection conn, including any disposing of any underlying file descriptors or related resources.

Once this function, no further access to the conn structure may be made.

Reset Connection State

void nng_http_reset(nng_http *conn);

The nng_http_reset function resets the request and response state of the the connection conn, so that it is just as if it had been freshly created with nng_http_client_connect or passed into a handler function for a server callback.

The intended purpose of this function is to clear the object state before reusing the conn for subsequent transactions.

Direct Read and Write

void nng_http_read(nng_http *conn, nng_aio *aio);
void nng_http_write(nng_http *conn, nng_aio *aio);
void nng_http_read_all(nng_http *conn, nng_aio *aio);
void nng_http_write_all(nng_http *conn, nng_aio *aio);

The nng_http_read and nng_http_write functions read or write data asynchronously from or to the connection conn, using the nng_iov that is set in aio with nng_aio_set_iov. These functions will complete as soon as any data is transferred. Use nng_aio_count to determine how much data was actually transferred.

The nng_http_read_all and nng_http_write_all functions perform the same task, but will keep resubmitting operations until the the entire amount of data requested by the nng_iov is transferred.

These functions are most likely to be useful after hijacking the connection with nng_http_hijack. They can be used to transfer request or response body data as well.

Hijacking Connections

nng_err nng_http_hijack(nng_http *conn);

TODO: This API will change to convert the conn into a stream object.

The nng_http_hijack function hijacks the connection conn, causing it to be disassociated from the HTTP server where it was created.

The purpose of this function is the creation of HTTP upgraders (such as WebSocket), where the underlying HTTP connection will be taken over for some other purpose, and should not be used any further by the server.

This function is most useful when called from a handler function. (See [nng_http_handler_alloc].)

Client API

The NNG client API consists of an API for creating connections, and an API for performing transactions on those connections.

Client Object

typedef struct nng_http_client nng_http_client;

The nng_http_client object is the client side creator for nng_http objects. It is analogous to a dialer object used elsewhere in NNG, but it specifically is only for HTTP.

Create a Client

void nng_http_client_alloc(nng_http_client *clientp, const nng_url *url);

The nng_http_client_alloc allocates an HTTP client suitable for connecting to the server identified by url and stores a pointer to it in the location referenced by clientp.

Destroy a Client

void nng_http_client_free(nng_http_client *client);

The nng_http_client_free connection destroys the client object and any of its resources.

Client TLS

nng_err nng_http_client_get_tls(nng_http_client *client, nng_tls_config **tlsp);
nng_err nng_http_client_set_tls(nng_http_client *client, nng_tls_config *tls);

The nng_http_client_get_tls and nng_http_client_set_tls functions are used to retrieve or change the TLS configuration used when making outbound connections, enabling TLS as a result.

If TLS has not been previously configured on client, then nng_http_client_get_tls will return NNG_EINVAL. Both functions will return NNG_ENOTSUP if either HTTP or TLS is not supported.

Calling nng_http_client_set_tls invalidates any client previously obtained with nng_http_client_get_tls, unless a separate hold on the object was obtained.

Once TLS is enabled for an nng_http_client, it is not possible to disable TLS.

Creating Connections

#include <nng/http.h>

void nng_http_client_connect(nng_http_client *client, nng_aio *aio);

The nng_http_client_connect function makes an outgoing connection to the server configured for client, and creates an nng_http object for the connection.

This is done asynchronously, and when the operation succeseds the connection may be retried from the aio using nng_aio_get_output with index 0.

Example 1: Connecting to Google

nng_aio *aio;
nng_url *url;
nng_http_client *client;
nng_http *conn;
nng_err rv;

// Error checks elided for clarity.
nng_url_parse(&url, "http://www.google.com");
nng_aio_alloc(&aio, NULL, NULL);
nng_http_client_alloc(&client, url);

nng_http_client_connect(client, aio);

// Wait for connection to establish (or attempt to fail).
nng_aio_wait(aio);

if ((rv = nng_aio_result(aio)) != 0) {
    printf("Connection failed: %s\n", nng_strerror(rv));
} else {
    // Connection established, get it.
    conn = nng_aio_get_output(aio, 0);

    // ... do something with it here

    // Close the connection when done to avoid leaking it.
    nng_http_close(conn);
}

Preparing a Transaction

Sending the Request

void nng_http_write_request(nng_http *conn, nng_aio *aio);

The nng_http_write_request function starts an asynchronous write of the HTTP request associated with conn. The entire request is sent, including headers, and if present, the request body data. (The request body can be set with [nng_http_set_data] or [nng_http_copy_data].)

This function returns immediately, with no return value. Completion of the operation is signaled via the aio, and the final result may be obtained via nng_aio_result.

Obtaining the Response

void nng_http_read_response(nng_http *conn, nng_aio *aio);

The nng_http_read_response function starts an asynchronous read from the HTTP connection conn, reading an HTTP response into the response associated with conn, including all of the related headers.

It does not transfer any response body. To do that, use nng_http_read_all or nng_http_read.

Submitting the Transaction

void nng_http_transact(nng_http *conn, nng_aio *aio);

The HTTP request is issued, and the response processed, asynchronously by the nng_http_transact function. When the function is complete, the aio will be notified.

The nng_http_transact function is used to perform a complete HTTP exchange over the connection conn, sending the request and attached body data to the remote server, and reading the response.

The entire response is read, including any associated body, which can subsequently be obtained using [nng_http_get_data].

This function is intended to make creation of client applications easier, by performing multiple asynchronous operations required to complete an entire HTTP transaction.

If an error occurs, the caller should close conn with nng_http_close, as it may not necessarily be usable with other transactions.

This function returns immediately, with no return value. Completion of the operation is signaled via the aio, and the final result may be obtained via [nng_aio_result()].

Response Body

Server API

Handlers

typedef struct nng_http_handler nng_http_handler;

An nng_http_handler encapsulates a function used used to handle incoming requests on an HTTP server, routed based on method and URI, and the parameters used with that function.

Every handler has a Request-URI to which it refers, which is determined by the path argument. Only the path component of the Request URI is considered when determining whether the handler should be called.

This implementation limits the path length to 1024 bytes, including the zero termination byte. This does not prevent requests with much longer URIs from being supported, but doing so will require setting the handler to match a parent path in the tree using [nng_http_handler_set_tree].

Additionally each handler has a method it is registered to handle (the default is “GET” andc can be changed with [nng_http_handler_set_method]), and optionally a “Host” header it can be matched against (see [nng_http_handler_set_host]).

In some cases, a handler may reference a logical tree rather (directory) rather than just a single element. (See [nng_http_handler_set_tree]).

Implementing a Handler

typedef void (*nng_http_hander_func)(nng_http_conn *conn, void *arg, nng_aio *aio);

nng_err nng_http_handler_alloc(nng_http_handler **hp, const char *path, nng_http_handler_func cb);

The nng_http_handler_alloc function allocates a generic handler which will be used to process requests coming into an HTTP server. On success, a pointer to the handler is stored at the located pointed to by hp.

The handler function is specified by cb. This function uses the asynchronous I/O framework.

The function receives the connection on conn, and an optional data pointer that was set previously with [nng_http_handler_set_data] as the second argument. The final argument is the nng_aio aio, which must be “finished” to complete the operation.

The handler may call [nng_http_write_response] to send the response, or it may simply let the framework do so on its behalf. The server will perform this step if the callback has not already done so.

Response headers may be set using nng_http_set_header, and request headers may be accessed by using nng_http_get_header.

Likewise the request body may be accessed, using nng_http_get_body, and the response body may be set using either nng_http_set_body or nng_http_copy_body.

The HTTP status should be set for the transaction using nng_http_set_status.

Finally, the handler should finish the operation by calling the nng_aio_finish function after having set the status to [NNG_OK]. If any other status is set on the aio, then a generic 500 response will be created and sent, if possible, and the connection will be closed.

The aio may be scheduled for deferred completion using the nng_aio_start.

Serving Directories and Files

nng_err nng_http_handler_alloc_directory(nng_http_handler **hp, const char *path, const char *dirname);
nng_err nng_http_handler_alloc_file(nng_http_handler **hp, const char *path, const char *filename);

The nng_http_handler_alloc_directory and nng_http_handler_alloc_file create handlers pre-configured to act as static content servers for either a full directory at dirname, or the single file at filename. These support the “GET” and “HEAD” methods, and the directory variant will dynamically generate index.html content based on the directory contents. These will also set the “Content-Type” if the file extension matches one of the built-in values already known. If the no suitable MIME type can be determined, the content type is set to “application/octet-stream”.

Static Handler

nng_err nng_http_handler_alloc_static(nng_http_handler **hp, const char *path,
        const void *data, size_t size, const char *content_type);

The nng_http_handler_alloc_static function creates a handler that serves the content located in data (consisting of size bytes) at the URI path. The content_type determines the “Content-Type” header. If NULL is specified then a value of application/octet-stream is assumed.

Redirect Handler

nng_err nng_http_handler_alloc_redirect(nng_http_handler **hp, const char *path,
        nng_http_status status, const char *location);

The nng_http_handler_alloc_redirect function creates a handler with a function that simply directions from the URI at path to the given location.

The HTTP reply it creates will be with [status code][nng_http_status] status, which should be a 3XX code such as 301, and a Location: header will contain the URL referenced by location, with any residual suffix from the request URI appended.

Collecting Request Body

void nng_http_handler_collect_body(nng_http_handler *handler, bool want, size_t maxsz);

The nng_http_handler_collect_body function requests that HTTP server framework collect any reuqest body for the request and attach it to the connection before calling the callback for the handler.

Subsequently the data can be retrieved by the handler from the request with the nng_http_get_body function.

The collection is enabled if want is true. Furthermore, the data that the client may sent is limited by the value of maxsz. If the client attempts to send more data than maxsz, then the request will be terminated with [NNG_HTTP_STATUS_CONTENT_TOO_LARGE] (413).

In order to provide an unlimited size, use (size_t)-1 for maxsz. The value 0 for maxsz can be used to prevent any data from being passed by the client.

The built-in handlers for files, directories, and static data limit the maxsz to zero by default. Otherwise the default setting is to enable this capability with a default value of maxsz of 1 megabyte.

Setting Callback Argument

void nng_http_handler_set_data(nng_http_handler *handler, void *data,
    void (*dtor)(void *));

The nng_http_handler_set_data function is used to set the data argument that will be passed to the callback.

Additionally, when the handler is deallocated, if dtor is not NULL, then it will be called with data as its argument. The intended use of this function is deallocate any resources associated with data.

Setting the Method

void nng_http_handler_set_method(nng_http_handler *handler, const char *method);

The nng_http_handler_set_method function sets the method that the handler will be called for, such as “GET” or “POST”. (By default the “GET” method is handled.)

If method is NULL the handler will be executed for all methods. The handler may determine the actual method used with the nng_http_get_method function.

The server will automatically call “GET” handlers if the client sends a “HEAD” request, and will suppress HTTP body data in the responses sent for such requests.

Filtering by Host

void nng_http_handler_set_host(nng_http_handler *handler, const char *host);

The nng_http_handler_set_host function is used to limit the scope of the handler so that it will only be called when the specified host matches the value of the Host: HTTP header.

This can be used to create servers with different content for different virtual hosts.

The value of the host can include a colon and port, and should match exactly the value of the Host header sent by the client. (Canonicalization of the host name is performed.)

Handling an Entire Tree

void nng_http_handler_set_tree(nng_http_handler *handler);

The nng_http_handler_set_tree function causes the handler to be matched if the request URI sent by the client is a logical child of the path for handler, and no more specific handler has been registered.

This is useful in cases when the handler would like to examine the entire path and possibly behave differently; for example a REST API that uses the rest of the path to pass additional parameters.

Sending the Response Explicitly

void nng_http_write_response(nng_http *conn, nng_aio *aio);

Normally the server will send any attached response, but there are circumstances where a response must be sent manually, such as when hijacking a connection.

In such a case, nng_http_write_response can be called, which will send the response and any attached data, asynchronously using the nng_aio aio.

By default, for HTTP/1.1 connections, the connection is kept open, and will be reused to receive new requests. For HTTP/1.0, or if the client has requested explicitly by setting the “Connection: close” header, the connection will be closed after the response is fully sent.

Miscellaneous

This chapter discusses some interfaces that don’t really fit anywhere else.

Get Random Number

uint32_t nng_random(void);

The nng_random returns a random number. The value returned is suitable for use with cryptographic functions such as key generation, and is obtained using platform-specific cryptographically strong random number facilities when available.

Create Socket Pair

nng_err nng_socket_pair(int fds[2]);

The nng_socket_pair function creates a pair of connected file descriptors. These file descriptors, which are returned in the fds array, are suitable for use with the Socket transport.

On POSIX platforms, this is a thin wrapper around the standard socketpair function, using the AF_UNIX family and the SOCK_STREAM socket type. ¹

This will return zero on success, or an error number. On platforms that lack this capability, such as Windows, it will return NNG_ENOTSUP.

Report Library Version

const char * nng_version(void);

The nng_version function returns a human readable version number for NNG, formatted as a NUL-terminated string.

Additionally, compile time version information is available via some predefined macros:

• NNG_MAJOR_VERSION: Major version number.
• NNG_MINOR_VERSION: Minor version number.
• NNG_PATCH_VERSION: Patch version number.

NNG is developed and released using Semantic Versioning 2.0, and the version numbers reported refer to both the API and the library itself. (The ABI – application binary interface – between the library and the application is controlled in a similar, but different manner depending upon the link options and how the library is built.)

1: At present only POSIX platforms implementing socketpair support this function.

ID Map

Internally, NNG uses a map of numeric identifiers to data structures. This feature is also exposed for application use, as a “supplemental” feature.

When using these functions, it is necessary to add the #include <nng/supplemental/util/idhash.h> include file to list of includes.

ID Map Structure

typedef struct nng_id_map_s nng_id_map;

The ID map structure, nng_id_map provides a table of identifiers mapping to user-supplied pointers (which must not be NULL). The identifiers can be thought of as indices into the table, with the pointers providing the reference for the user supplied data.

The values of identifiers can be supplied by the user, or can be allocated automatically by nng_id_map from a predefined range. The starting point for allocations can also be randomly within the range.

The identifiers are 64-bit unsigned integers and can be sparse; the structure will use space efficiently even if identifiers are very far apart. ¹

Create ID Map

#define NNG_MAP_RANDOM 1

int nng_id_map_alloc(nng_id_map **map_p, uint64_t lo, uint64_t hi, int flags);

The nng_id_map_alloc function allocates a map without any data in it, and returns a pointer to it in map_p. When allocating identifiers dynamically, the values will be chosen from the range defined by lo and hi, inclusive.

The flags argument is a bit mask of flags that can adjust behavior of the map. The only flag defined at present is NNG_MAP_RANDOM, which causes the first identifier allocation to start at a random point within the range. This is useful to reduce the odds of different instances of an application using the same identifiers at the same time.

If both lo and hi are zero, then the values 0 and 0xffffffff are substituted in their place, giving a full range of 32-bit identifiers.

This function can return NNG_ENOMEM if it is unable to allocate resources, otherwise it returns zero on success.

Destroy Map

void nng_id_map_free(nng_id_map *map);

The nng_id_map_free function destroys map, releasing any resources associated with it.

Store a Value

int nng_id_set(nng_id_map *map, uint64_t id, void *value);

The nng_id_map_set function is used to store the value in the map at index id.

If another value is already stored at that same location, then it is overwritten with value.

If the table has to grow to accommodate this value, it may fail if insufficient memory is available, returning NNG_ENOMEM. Otherwise it returns zero.

Lookup a Value

void *nng_id_get(nng_id_map *map, uint64_t id);

The nng_id_get function looks up the entry for id in map, returning the associated value if present, or NULL if no such entry exists.

Allocate an ID

int nng_id_alloc(nng_id_map *map, uint64_t *id_p, void *value);

The nng_id_alloc stores the value in the map, at a newly allocated index, and returns the index in id_p.

Identifiers are allocated in increasing order, without reusing old identifiers until the largest possible identifier is allocated. After wrapping, only identifiers that are no longer in use will be considered. No effort is made to order the availability of identifiers based on when they were freed.²

As with nng_id_set, this may need to allocate memory and can thus fail with NNG_ENOMEM.

Additionally, if there are no more free identifiers within the range specified when map was created, then it will return NNG_ENOSPC.

Otherwise it returns zero, indicating success.

Remove an ID

int nng_id_remove(nng_id_map *map, uint64_t id);

The nng_id_remove removes the entry at index id from map.

If no such entry exist, it will return NNG_ENOENT. Otherwise it returns zero.

Iterating IDs

bool nng_id_visit(nng_id_map *map, uint64_t *id_p, void **value_p, uint32_t *cursor);

The nng_id_visit function is used to iterate over all items in the table. The caller starts the iteration by setting the cursor to 0 before calling it. For each call, the associated key and value of the next item will be returned in id_p, and value_p and the cursor will be updated. When all items have been iterated, the function returns false. The order of items returned is not guaranteed to be sequential. The caller must not attempt to derive any value of the cursor as it refers to internal table indices.

Entries may be safely removed from map while iterating.

However, if new entries are added to the table while iterating, the result of iteration is undefined; entries may be repeated or omitted during such an iteration.

The caller must not attempt to derive any value of the cursor as it refers to internal table indices.

1: The ID map is capable of storing at most 2³² identifiers, even though the identifers may themselves be much larger than this.

2: The concern about possibly reusing a recently released identifier comes into consideration after the range has wrapped. Given a sufficiently large range, this is unlikely to be a concern.

Command Arguments

Some NNG utilities need to parse command line options, and for this purpose a header library is supplied.

To make use of this, the header <nng/args.h> must be included.

Parse Command Line Arguments

typedef struct nng_arg_spec {
    const char *a_name;  // Long style name (may be NULL for short only)
    int         a_short; // Short option (no clustering!)
    int         a_val;   // Value stored on a good parse (>0)
    bool        a_arg;   // Option takes an argument if true
} nng_optspec;

#define NNG_ARG_END     (-1)
#define NNG_ARG_INVAL   (-2)
#define NNG_ARG_AMBIG   (-3)
#define NNG_ARG_MISSING (-4)

int nng_args_parse(int argc, char *const *argv,
                   const nng_optspec *spec, int *val, char **arg, int *idx);

The nng_args_parse function is intended to facilitate parsing command-line arguments. This function exists largely to stand in for getopt from POSIX systems, but it is available on all platforms, and it includes some capabilities missing from getopt.

The function parses arguments from main¹ (using argc and argv), starting at the index referenced by idx. (New invocations typically set the value pointed to by idx to 1.)

Options are parsed as specified by spec (see Argument Specification.) The value of the parsed option will be stored at the address indicated by val, and the value of idx will be incremented to reflect the next option to parse.

If the option had an argument, a pointer to that is returned at the address referenced by arg.

This function should be called repeatedly, until it returns either -1 (indicating the end of options is reached) or a non-zero error code is returned.

This function may return the following errors:

• [NNG_ARG_AMBIGU]: Parsed option matches more than one specification.
• [NNG_ARG_MISSING]: Option requires an argument, but one is not present.
• [NNG_ARG_INVAL]: An invalid (unknown) argument is present in argv.

Option Specification

The calling program must first create an array of nng_arg_spec structures describing the options to be supported. This structure has the following members:

• a_name:

  The long style name for the option, such as “verbose”. This will be parsed as a long option on the command line when it is prefixed with two dashes. It may be NULL if only a short option is to be supported.

• a_short:

  This is a single letter (at present only ASCII letters are supported). These options appear as just a single letter, and are prefixed with a single dash on the command line. The use of a slash in lieu of the dash is not supported, in order to avoid confusion with path name arguments. This value may be set to 0 if no short option is needed.

• o_val:

  This is a numeric value that is unique to this option. This value is assigned by the application program, and must be non-zero for a valid option. If this is zero, then it indicates the end of the specifications, and the rest of this structure is ignored. The value will be returned to the caller in val by nng_args_parse when this option is parsed from the command line.

• a_arg:

  This value should be set to true if the option should take an argument.

Long Options

Long options are parsed from the argv array, and are indicated when the element being scanned starts with two dashes. For example, the “verbose” option would be specified as --verbose on the command line. If a long option takes an argument, it can either immediately follow the option as the next element in argv, or it can be appended to the option, separated from the option by an equals sign (=) or a colon (:).

Short Options

Short options appear by themselves in an argv element, prefixed by a dash (-). If the short option takes an argument, it can either be appended in the same element of argv, or may appear in the next argv element.

Prefix Matching

When using long options, the parser will match if it is equal to a prefix of the a_name member of a option specification, provided that it do so unambiguously (meaning it must not match any other option specification.)

Example

The following program fragment demonstrates this function.

    enum { OPT_LOGFILE, OPT_VERBOSE };
    char *logfile; // options to be set
    bool verbose;

    static nng_arg_spec specs[] = {
        {
            .a_name = "logfile",
            .a_short = 'D',
            .a_val = OPT_LOGFILE,
            .a_arg = true,
        }, {
            .a_name = "verbose",
            .a_short = 'V',
            .a_val = OPT_VERBOSE,
            .a_arg = false,
        }, {
            .a_val = 0; // Terminate array
        }
    };

    for (int idx = 1;;) {
        int rv, opt;
        char *arg;
        rv = nng_args_parse(argc, argv, specs, &opt, &arg, &idx);
        if (rv != 0) {
            break;
        }
        switch (opt) {
        case OPT_LOGFILE:
            logfile = arg;
            break;
        case OPT_VERBOSE:
            verbose = true;
            break;
        }
    }
    if (rv != NNG_ARG_END) {
        switch (rv) {
        case NNG_ARG_AMBIG:
            printf("Options error: ambiguous option\n");
            break;
        case NNG_ARG_MISSING:
            printf("Options error: required option argument missing\n");
            break;
        case NNG_ARG_INVAL:
            printf("Options error: unknown option present\n");
            break;
        }
        exit(1);
    }

1: Parsing argument strings from other sources can be done as well, although usually then idx will be initialized to zero.

Protocols

The Scalability Protocols are a principally a collection of common networking patterns found in applications.

The following patterns are included:

Request - Reply

The request/reply pattern is made up of the REQ and REP protocols. This most often used when implementing RPC-like services, where a given request is matched by a single reply.

Pipeline

The pipeline pattern is made up of the PUSH and PULL protocols.

In this pattern communication is half-duplex, in that one side sends data and another side receives.

This pattern is also characterized by its ability to solve distribution problems, and the fact that it has back-pressure, providing a measure of flow control to data production and consumption.

Publish - Subscribe

Bus

Pair

BUS Protocol

The BUS protocol provides for building mesh networks where every peer is connected to every other peer. In this protocol, each message sent by a node is sent to every one of its directly connected peers.

All message delivery in this pattern is best-effort, which means that peers may not receive messages. Furthermore, delivery may occur to some, all, or none of the directly connected peers. (Messages are not delivered when peer nodes are unable to receive.) Hence, send operations will never block; instead if the message cannot be delivered for any reason it is discarded.

Socket Operations

The nng_bus0_open functions create a bus socket. This socket may be used to send and receive messages. Sending messages will attempt to deliver to each directly connected peer.

Protocol Versions

Only version 0 of this protocol is supported. (At the time of writing, no other versions of this protocol have been defined.)

Protocol Options

The BUS protocol has no protocol-specific options.

Protocol Headers

When using a BUS socket in raw mode, received messages will contain the incoming pipe ID as the sole element in the header. If a message containing such a header is sent using a raw BUS socket, then, the message will be delivered to all connected pipes except the one identified in the header. This behavior is intended for use with device configurations consisting of just a single socket. Such configurations are useful in the creation of rebroadcasters, and this capability prevents a message from being routed back to its source. If no header is present, then a message is sent to all connected pipes.

When using normal (cooked mode) BUS sockets, no message headers are present.

PAIR protocol

The PAIR protocol implements a peer-to-peer pattern, where relationships between peers are one-to-one.

Socket Operations

The nng_pair_open functions create a PAIR socket.

Normally, this pattern will block when attempting to send a message if no peer is able to receive the message.

Protocol Versions

Version 0 is the legacy version of this protocol. It lacks any header information, and is suitable when building simple one-to-one topologies.

Version 1 of the protocol offers improved protection against loops when used with devices.

Polyamorous Mode

Normally pair sockets are for one-to-one communication, and a given peer will reject new connections if it already has an active connection to another peer.

Polyamorous changes this, to allow a socket to communicate with multiple directly-connected peers. This mode is enabled by opening a socket using nng_pair1_open_poly.

In polyamorous mode a socket can support many one-to-one connections. In this mode, the application must choose the remote peer to receive an outgoing message by setting the nng_pipe to use for the outgoing message using nng_msg_set_pipe.

If no remote peer is specified by the sender, then the protocol will select any available connected peer.

Most often the value of the outgoing pipe will be obtained from an incoming message using nng_msg_get_pipe, such as when replying to an incoming message.

In order to prevent head-of-line blocking, if the peer on the given pipe is not able to receive (or the pipe is no longer available, such as if the peer has disconnected), then the message will be discarded with no notification to the sender.

Protocol Options

The following protocol-specific options are available.

• NNG_OPT_MAXTTL: (int, version 1 only). Maximum time-to-live.

• NNG_OPT_PAIR1_POLY: (bool, version 1 only) This option is no longer supported. Formerly it was used to configure polyamorous mode, but that mode is now established by using the nng_pair1_open_poly function.

Protocol Headers

Version 0 of the pair protocol has no protocol-specific headers.

Version 1 of the pair protocol uses a single 32-bit unsigned value. The low-order (big-endian) byte of this value contains a “hop” count, and is used in conjunction with the NNG_OPT_MAXTTL option to guard against device forwarding loops. This value is initialized to 1, and incremented each time the message is received by a new node.

PUB Protocol

The PUB protocol is one half of a publisher/subscriber pattern. In this pattern, a publisher sends data, which is broadcast to all subscribers. The subscribing applications only see the data to which they have subscribed.

The PUB protocol is the publisher side, and the SUB protocol is the subscriber side.

The topics that subscribers subscribe to is just the first part of the message body. Applications should construct their messages accordingly.

Socket Operations

The nng_pub0_open functions create a publisher socket. This socket may be used to send messages, but is unable to receive them. Attempts to receive messages will result in NNG_ENOTSUP.

Protocol Versions

Only version 0 of this protocol is supported. (At the time of writing, no other versions of this protocol have been defined.)

Protocol Options

The PUB protocol has no protocol-specific options.

Protocol Headers

The PUB protocol has no protocol-specific headers.

PULL protocol

The PULL protocol is one half of a pipeline pattern. The other half is the PUSH protocol.

In the pipeline pattern, pushers distribute messages to pullers. Each message sent by a pusher will be sent to one of its peer pullers, chosen in a round-robin fashion from the set of connected peers available for receiving. This property makes this pattern useful in load-balancing scenarios.

Socket Operations

The nng_pull0_open functions create a PULL socket. This socket may be used to receive messages, but is unable to send them. Attempts to send messages will result in NNG_ENOTSUP.

When receiving messages, the PULL protocol accepts messages as they arrive from peers. If two peers both have a message ready, the order in which messages are handled is undefined.

Protocol Versions

Only version 0 of this protocol is supported. (At the time of writing, no other versions of this protocol have been defined.)

Protocol Options

The PULL protocol has no protocol-specific options.

Protocol Headers

The PULL protocol has no protocol-specific headers.

PUSH protocol

DESCRIPTION

The PUSH protocol is one half of a pipeline pattern. The other side is the PULL protocol.

In the pipeline pattern, pushers distribute messages to pullers. Each message sent by a pusher will be sent to one of its peer pullers, chosen in a round-robin fashion from the set of connected peers available for receiving. This property makes this pattern useful in load-balancing scenarios.

Socket Operations

The nng_push0_open call creates a PUSH socket. This socket may be used to send messages, but is unable to receive them. Attempts to receive messages will result in NNG_ENOTSUP.

Send operations will observe flow control (back-pressure), so that only peers capable of accepting a message will be considered. If no peer is available to receive a message, then the send operation will wait until one is available, or the operation times out.

Protocol Versions

Only version 0 of this protocol is supported. (At the time of writing, no other versions of this protocol have been defined.)

Protocol Options

• NNG_OPT_SENDBUF: (int, 0 - 8192) Normally this is set to zero, indicating that send operations are unbuffered. In unbuffered operation, send operations will wait until a suitable peer is available to receive the message. If this is set to a positive value (up to 8192), then an intermediate buffer is provided for the socket with the specified depth (in messages).

Protocol Headers

The PUSH protocol has no protocol-specific headers.

REP Protocol

The REP protocol is one half of a request/reply pattern. In this pattern, a requester sends a message to one replier, who is expected to reply. The request is resent if no reply arrives, until a reply is received or the request times out.

The REP protocol is the replier side, and the REP protocol is the requester side.

Socket Operations

The nng_rep0_open functions create a replier socket. This socket may be used to receive messages (requests), and then to send replies.

Generally a reply can only be sent after receiving a request.

Send operations will result in NNG_ESTATE if no corresponding request was previously received.

Likewise, only one receive operation may be pending at a time. Any additional concurrent receive operations will result in NNG_ESTATE.

Raw mode sockets ignore all these restrictions.

Context Operations

This protocol supports the creation of contexts for concurrent use cases using nng_ctx_open.

Each context may have at most one outstanding request, and operates independently of the others. The restrictions for order of operations with sockets apply equally well for contexts, except that each context will be treated as if it were a separate socket.

Protocol Versions

Only version 0 of this protocol is supported. (At the time of writing, no other versions of this protocol have been defined.)

Protocol Options

The REP protocol has no protocol-specific options.

Protocol Headers

The REP protocol uses a backtrace in the header. This is more fully documented in the REQ chapter.

REQ protocol

The REQ protocol is one half of a request/reply pattern. In this pattern, a requester sends a message to one replier, who is expected to reply. The request is resent if no reply arrives, until a reply is received or the request times out.

The requester generally only has one outstanding request at a time unless in raw mode, and it will generally attempt to spread work requests to different peer repliers.

The REQ protocol is the requester side, and the REP protocol is the replier side.

Socket Operations

The nng_req0_open functions create a REQ socket. This socket may be used to send messages (requests), and then to receive replies.

Generally a reply can only be received after sending a request. (Attempts to receive a message will result in NNG_ESTATE if there is no outstanding request.)

Furthermore, only a single receive operation may be pending at a time. Attempts to post more receive operations concurrently will result in NNG_ESTATE.

Requests may be canceled by sending a different request. This will cause the requester to discard any reply from the earlier request, but it will not stop a replier from processing a request it has already received or terminate a request that has already been placed on the wire.

Raw mode sockets ignore all these restrictions.

Context Operations

This protocol supports the creation of contexts for concurrent use cases using nng_ctx_open.

The NNG_OPT_REQ_RESENDTIME value may be configured differently on contexts created this way.

Each context may have at most one outstanding request, and operates independently from the others.

The restrictions for order of operations with sockets apply equally well for contexts, except that each context will be treated as if it were a separate socket.

Protocol Versions

Only version 0 of this protocol is supported. (At the time of writing, no other versions of this protocol have been defined.)

Protocol Options

The following protocol-specific option is available.

• NNG_OPT_REQ_RESENDTIME:
  (nng_duration)
  When a new request is started, a timer of this duration is also started. If no reply is received before this timer expires, then the request will be resent.
  
  Requests are also automatically resent if the peer to whom the original request was sent disconnects.
  
  Resending may be deferred up to the value of the NNG_OPT_RESENDTICK parameter.
  
  If the value is set to NNG_DURATION_INFINITE, then resends are disabled altogether. This should be used when the request is not idemptoent.

• NNG_OPT_REQ_RESENDTICK:
  (nng_duration)
  This is the granularity of the clock that is used to check for resending. The default is a second. Setting this to a higher rate will allow for more timely resending to occur, but may incur significant additional overhead when the socket has many outstanding requests (contexts).
  
  When there are no requests outstanding that have a resend set, then the clock does not tick at all.
  
  This option is shared for all contexts on a socket, and is only available for the socket itself.

Protocol Headers

This protocol uses a backtrace in the header. This form uses a stack of 32-bit big-endian identifiers. There must be at least one identifier, the request ID, which will be the last element in the array, and must have the most significant bit set.

There may be additional peer IDs preceding the request ID. These will be distinguishable from the request ID by having their most significant bit clear.

When a request message is received by a forwarding node (such as a device), the forwarding node prepends a 32-bit peer ID (which must have the most significant bit clear), which is the forwarder’s way of identifying the directly connected peer from which it received the message. (This peer ID, except for the most significant bit, has meaning only to the forwarding node itself.)

It may help to think of prepending a peer ID as pushing a peer ID onto the front of the stack of headers for the message. (It will use the peer ID it popped from the front to determine the next intermediate destination for the reply.)

When a reply message is created, it is created using the same headers that the request contained.

A forwarding node can pop the peer ID it originally pushed on the message, stripping it from the front of the message as it does so.

When the reply finally arrives back at the initiating requester, it should have only a single element in the message, which will be the request ID it originally used for the request.

RESPONDENT protocol

The RESPONDENT protocol is one half of a survey pattern. In this pattern, a surveyor sends a survey, which is broadcast to all peer respondents. The respondents then have a chance to reply (but are not obliged to reply). The survey itself is a timed event, so that responses received after the survey has finished are discarded.

The RESPONDENT protocol is the respondent side, and the SURVEYOR protocol is the surveyor side.

Socket Operations

The nng_respondent0_open functions create a respondent socket. This socket may be used to receive messages, and then to send replies. A reply can only be sent after receiving a survey, and generally the reply will be sent to surveyor from whom the last survey was received.

Respondents may discard a survey by simply not replying to it.

Raw mode sockets ignore all these restrictions.

Context Operations

This protocol supports the creation of contexts for concurrent use cases using nng_ctx_open.

Incoming surveys will be routed to and received by only one context. Additional surveys may be received by other contexts in parallel. Replies made using a context will be returned to the the surveyor that issued the survey most recently received by that context. The restrictions for order of operations with sockets apply equally well for contexts, except that each context will be treated as if it were a separate socket.

Protocol Versions

Only version 0 of this protocol is supported. At the time of writing, no other versions of this protocol have been defined. ¹

Protocol Options

The respondent protocol has no protocol-specific options.

Protocol Headers

The RESPONDENT protocol uses a backtrace in the header. This is more fully documented in the SURVEYOR manual.

1: An earlier and incompatible version of the protocol was used in older pre-releases of nanomsg, but was not released in any production version.

SUB protocol

The SUB protocol is one half of a publisher/subscriber pattern. In this pattern, a publisher sends data, which is broadcast to all subscribers. The subscribing applications only see the data to which they have subscribed.

The SUB protocol is the subscriber side, and the PUB protocol is the publisher side.

The topics that subscribers subscribe to is compared to the leading bytes of the message body. Applications should construct their messages accordingly.

Socket Operations

The nng_sub0_open functions create a SUB socket. This socket may be used to receive messages, but is unable to send them. Attempts to send messages will result in NNG_ENOTSUP.

Protocol Versions

Only version 0 of this protocol is supported. (At the time of writing, no other versions of this protocol have been defined.)

Protocol Options

The following protocol-specific option is available.

• NNG_OPT_SUB_PREFNEW:
  (bool)
  
  This read/write option specifies the behavior of the subscriber when the queue is full. When true (the default), the subscriber will make room in the queue by removing the oldest message. When false, the subscriber will reject messages if the message queue does not have room.

Protocol Headers

The SUB protocol has no protocol-specific headers.

SURVEYOR protocol

The SURVEYOR protocol is one half of a survey pattern. In this pattern, a surveyor sends a survey, which is broadcast to all peer respondents. The respondents then have a chance, but are not obliged, to reply. The survey itself is a timed event, so that responses received after the survey has finished are discarded.

The SURVEYOR protocol is the surveyor side, and the RESPONDENT protocol is the respondent side.

Socket Operations

The nng_surveyor0_open functions create a surveyor socket. This socket may be used to send messages (surveys), and then to receive replies. A reply can only be received after sending a survey. A surveyor can normally expect to receive at most one reply from each responder. (Messages can be duplicated in some topologies, so there is no guarantee of this.)

Attempts to receive on a socket with no outstanding survey will result in NNG_ESTATE. If the survey times out while the surveyor is waiting for replies, then the result will be NNG_ETIMEDOUT.

Only one survey can be outstanding at a time; sending another survey will cancel the prior one, and any responses from respondents from the prior survey that arrive after this will be discarded.

Raw mode sockets ignore all these restrictions.

Context Operations

This protocol supports the creation of contexts for concurrent use cases using nng_ctx_open.

Each context can initiate its own surveys, and it will receive only responses to its own outstanding surveys. Other contexts on the same socket may have overlapping surveys operating at the same time.

Each of these may have their own timeouts established with NNG_OPT_SURVEYOR_SURVEYTIME.

Additionally, sending a survey on a context will only cancel an outstanding survey on the same context.

Protocol Versions

Only version 0 of this protocol is supported. At the time of writing, no other versions of this protocol have been defined. ¹

Protocol Options

The following protocol-specific option is available.

• NNG_OPT_SURVEYOR_SURVEYTIME:
  (nng_duration)
  
  When a new survey is started, a timer of this duration is started. Any responses arriving this time will be discarded. Attempts to receive after the timer expires with no other surveys started will result in NNG_ESTATE.
  
  If a receive is pending when this timer expires, it will result in NNG_ETIMEDOUT.

Protocol Headers

This form uses a stack of 32-bit big-endian identifiers. There must be at least one identifier, the survey ID, which will be the last element in the array, and must have the most significant bit set.

There may be additional peer IDs preceding the survey ID. These will be distinguishable from the survey ID by having their most significant bit clear.

When a survey message is received by a forwarding node (such as a device), the forwarding node prepends a 32-bit peer ID (which must have the most significant bit clear), which is the forwarder’s way of identifying the directly connected peer from which it received the message. (This peer ID, except for the most significant bit, has meaning only to the forwarding node itself.)

It may help to think of prepending a peer ID as pushing a peer ID onto the front of the stack of headers for the message. (It will use the peer ID it popped from the front to determine the next intermediate destination for the response.)

When a response message is created, it is created using the same headers that the survey contained.

A forwarding node can pop the peer ID it originally pushed on the message, stripping it from the front of the message as it does so.

When the response finally arrives back at the initiating surveyor, it should have only a single element in the message, which will be the survey ID it originally used for the request.

More detail can be found in the sp-surveyor-01 RFC document.

1: An earlier and incompatible version of the protocol was used in older pre-releases of nanomsg, but was not released in any production version.

Transports

This section documents transports for Scalabity Protocols implemented by NNG.

• BSD Socket
• Intra-Process Transport
• UDP

INPROC Transport

The inproc transportintra-process provides communication support between sockets within the same process. This may be used as an alternative to slower transports when data must be moved within the same process.

This transport tries hard to avoid copying data, and thus is very light-weight.

URL Format

This transport uses URLs using the scheme inproc://, followed by an arbitrary string of text, terminated by a NUL byte.

Multiple URLs can be used within the same application, and they will not interfere with one another.

Two applications may also use the same URL without interfering with each other. They will however be unable to communicate with each other using that URL.

Socket Address

When using an nng_sockaddr structure, the actual structure is of type nng_sockaddr_inproc.

Transport Options

The inproc transport has no special options.

Mixing Implementations

When mixing the NNG library with other implementations of these protocols in the same process (such as the mangos or libnanomsg implementations), it will not be possible to utilize the inproc transport to communicate across this boundary.

This limitation also extends to using different instances of the NNG library within the same process.

IPC Transport

DESCRIPTION

The ipc transport provides communication support between sockets within different processes on the same host. For POSIX platforms, this is implemented using UNIX domain sockets. For Windows, this is implemented using Windows named pipes. Other platforms may have different implementation strategies.

URI Formats

Traditional Names

This transport uses URIs using the scheme ipc://, followed by a path name in the file system where the socket or named pipe should be created.

UNIX Aliases

The unix:// scheme is an alias for ipc:// and can be used inter-changeably, but only on POSIX systems. ¹

Abstract Names

On Linux, this transport also can support abstract sockets. Abstract sockets use a URI-encoded name after the abstract:// scheme, which allows arbitrary values to be conveyed in the path, including embedded NUL bytes. For example, the name "a\0b" would be represented as abstract://a%00b.

Abstract names do not include the leading NUL byte used in the low-level socket address.

Abstract sockets do not have any representation in the file system, and are automatically freed by the system when no longer in use.

Abstract sockets ignore socket permissions, but it is still possible to determine the credentials of the peer with NNG_OPT_PEER_UID, and similar options. ²

Socket Address

When using an nng_sockaddr structure, the actual structure is of type nng_sockaddr_ipc, except for abstract sockets, which use nng_sockaddr_abstract.

Transport Options

The following transport options are supported by this transport, where supported by the underlying platform.

Other Configuration Parameters

On Windows systems, the security descriptor for the listener, which can be used to control access, can be set using the function nng_listener_set_security_descriptor.

1: The purpose of this scheme is to support a future transport making use of AF_UNIX on Windows systems, at which time it will be necessary to discriminate between the Named Pipes and the AF_UNIX based transports.

2: This property makes it important that names be chosen randomly to prevent unauthorized access, or that checks against the peer credentials are made, or ideally, both.

Socket Transport (Experimental)

Description

The socket transport supports communication between peers across arbitrary BSD sockets, such as those that are created with nng_socket_pair.

This transport only supports listeners, using nng_listener_create.

The socket file descriptor is passed to the listener using the NNG_OPT_SOCKET_FD option (as an integer). Setting this option will cause the listener to create a pipe backed by the file descriptor.

The protocol between peers using this transport is compatible with the protocol used for the tcp transport, but this is an implementation detail and subject to change without notice. ¹

URL Format

This transport uses the URL socket://, without further qualification.

Socket Address

The socket address will be of family NNG_AF_UNSPEC. There are no further socket details available.

Transport Options

The following transport options are supported by this transport.

1: Specifically it is not compatible with the ipc transport.

2: Windows lacks a suitable socketpair equivalent function we could use.

UDP Transport (Experimental)

Description

The udp transport supports communication between peers using UDP.

UDP is a very light-weight connection-less, unreliable, unordered delivery mechanism.

Both IPv4 and IPv6 are supported when the underlying platform also supports it.

This transport adds an ordering guarantee, so that messages will always be received in the correct order. Messages that arrive out of order, or are duplicated, will be dropped. There may be gaps in the messages received, so applications should not assume that all messages sent will arrive.

URL Format

This transport uses URIs using the scheme udp://, followed by an IP address or hostname, followed by a colon and finally a UDP port number. For example, to contact port 8001 on the localhost either of the following URIs could be used: udp://127.0.0.1:8001 or udp://localhost:8001.

A URI may be restricted to IPv6 using the scheme udp6://, and may be restricted to IPv4 using the scheme udp4://.

When specifying IPv6 addresses, the address must be enclosed in square brackets ([]) to avoid confusion with the final colon separating the port.

For example, the same port 8001 on the IPv6 loopback address (::1) would be specified as udp://[::1]:8001.

The special value of 0 (INADDR_ANY) can be used for a listener to indicate that it should listen on all interfaces on the host. A short-hand for this form is to omit the IP address entirely. For example, the following two URIs are equivalent, and could be used to listen to port 9999 on the host:

1. udp://0.0.0.0:9999
2. udp://:9999

Socket Address

When using an nng_sockaddr structure, the actual structure is either of type nng_sockaddr_in (for IPv4) or nng_sockaddr_in6 (for IPv6).

Transport Options

The following transport options are supported by this transport, where supported by the underlying platform.

Maximum Message Size

This transport maps each SP message to a single UDP packet. In order to allow room for network headers, we thus limit the maximum message size to 65000 bytes, minus the overhead for any SP protocol headers.

However, applications are strongly encouraged to only use this transport for very much smaller messages, ideally those that will fit within a single network packet without requiring fragmentation and reassembly.

For Ethernet without jumbo frames, this typically means an MTU of a little less than 1500 bytes. (Specifically, 1452, which allows 28 bytes for IPv4 and UDP, and 20 bytes for the this transport. Reduce by an additional 20 bytes for IPv6.)

Other link layers may have different MTUs, however IPv6 requires a minimum MTU of 1280, which after deducting 48 bytes for IPv6 and UDP headers, and 20 bytes for our transport header, leaves 1212 bytes for user data. If additional allowances are made for SP protocol headers with a default TTL of 8 (resulting in 72 additional bytes for route information), the final user accessible payload will be 1140 bytes. Thus this can be likely be viewed as a safe maximum to employ for SP payload data across all transports.

The maximum message size is negotiated as part of establishing a peering relationship, and oversize messages will be dropped by the sender before going to the network.

The maximum message size to receive can be configured with the NNG_OPT_RECVMAXSZ option.

Keep Alive

This transports maintains a logical “connection” with each peer, to provide a rough facsimile of a connection based semantic. This requires some resource on each peer. In order to ensure that resources are reclaimed when a peer vanishes unexpectedly, a keep-alive mechanism is implemented.

TODO: Document the tunables for this.

Migration Guides

This section provides some guides to aid in migrating software from earlier versions NNG or libnanomsg.

• Migrating from NNG 1.x
• Migrating from libnanomsg

Migrating from NNG 1.x

There are some incompatibities from NNG 1.x, and applications must make certain changes for NNG 2.0. This guide should help with this migration. While we have made reasonable efforts to highlight all of the things that applications could run into, this list is not necessarily exhaustive, and undocumented interfaces may have changed without notice here.

Detecting NNG v2

For applications that need to detect NNG v2 versus older code, the NNG_MAJOR_VERSION macro can be used. This will have numeric value 2 for version 2, and 1 for earlier versions.

Nanomsg Compatibility

Applications using the legacy libnanomsg API will have to be updated to native NNG interfaces. See the Migrating From libnanomsg chapter for details.

Library Initialization

It is now required for applications to initialize the library explicitly before using it. This is done using the nng_init function.

Removed Headers

The following header files are removed, and the declarations they provided are now provided by including <nng/nng.h>. Simply remove any references to them.

• nng/protocol/bus0/bus.h
• nng/protocol/pair0/pair.h
• nng/protocol/pair1/pair.h
• nng/protocol/pipeline0/pull.h
• nng/protocol/pipeline0/push.h
• nng/protocol/pubsub0/pub.h
• nng/protocol/pubsub0/sub.h
• nng/protocol/reqrep0/rep.h
• nng/protocol/reqrep0/req.h
• nng/protocol/survey0/respond.h
• nng/protocol/survey0/survey.h
• nng/supplemental/tls/tls.h
• nng/supplemental/util/idhash.h
• nng/supplemental/util/platform.h
• nng/transport/inproc/inproc.h
• nng/transport/ipc/ipc.h
• nng/transport/tcp/tcp.h
• nng/transport/tls/tls.h
• nng/transport/ws/websocket.h
• nng/transport/zerotier/zerotier.h

Renamed Functions

The following functions have been renamed as described by the following table. The old names are available by defining the macro NNG1_TRANSITION in your compilation environment.

Removed Protocol Aliases

The following macro aliases are removed, unless NNG1_TRANSITION is defined in your compilation environment.

• nng_bus_open
• nng_pair_open
• nng_pub_open
• nng_pull_open
• nng_push_open
• nng_rep_open
• nng_req_open
• nng_respondent_open
• nng_sub_open
• nng_surveyor_open

Just add either 0 or 1 (in the case of PAIRv1) to get the protocol desired. (Forcing the version number to be supplied should avoid surprises later as new versions of protocols are added.)

NNG_FLAG_ALLOC Removed

The NNG_FLAG_ALLOC flag that allowed a zero copy semantic with nng_send and nng_recv is removed. This was implemented mostly to aid legacy nanomsg applications, and it was both error prone and still a bit suboptimal in terms of performance.

Modern code should use one of nng_sendmsg, nng_recvmsg, nng_socket_send, or nng_socket_recv to get the maximum performance benefit. Working directly with nng_msg structures gives more control, reduces copies, and reduces allocation activity.

Error Code Changes

When an operation fails with NNG_ESTOPPED, it means that the associated [nni_aio] object has been permanently stopped and must not be reused. Applications must watch for this error code, and not resubmit an operation that returns it. This is particularly important for callbacks that automatically resubmit operations. Failure to observe this rule will lead to an infinite loop as any further operations on the object will fail immediately with NNG_ESTOPPED.

The error codes NNG_EAMBIGUOUS and NNG_ENOARG have been removed.

AIO Provider API changes

The API used for providers for asynchronous I/O operations has changed slightly.

• The nng_aio_begin function is removed. However a new nng_aio_reset function should be called instead, before performing any other operations on an aio object. (This simply clears certain fields.)
• The nng_aio_defer function is replaced, with a very nng_aio_start function. However, this function has slightly different semantics. It will automatically call the callback if the operation cannot be scheduled.
• Be aware of the new NNG_ESTOPPED error code, for operations on a handle that is being torn down by the consumer.

Transport Specific Functions

Transports have not needed to be registered for a long time now, and the functions for doing so have been removed. These functions can be simply removed from your application:

• nng_inproc_register
• nng_ipc_register
• nng_tls_register
• nng_tcp_register
• nng_ws_register
• nng_wss_register
• nng_zt_register

Additionally, the header files containing these functions have been removed, such as nng/transport/ipc/ipc.h. Simply remove #include references to those files.

TLS Configuration

The support for configuring TLS via NNG_OPT_TLS_CONFIG, NNG_TLS_AUTH_MODE, NNG_OPT_TLS_CA_FILE, NNG_OPT_TLS_SERVER_NAME, and similar has been removed.

Instead configuration must be performed by allocating a nng_tls_config object, and then setting fields on it using the appropriate functions, after which it may be configured on a listener or dialer using the nng_listener_set_tls or nng_dialer_set_tls functions.

Likewise, when using the streams API, use the nng_stream_listener_set_tls or nng_stream_dialer_set_tls functions.

Note that the declarations needed for TLS configuration are now available in <nng/nng.h>, rather than the supplemental header.

Old TLS Versions Removed

Support for very old TLS versions 1.0 and 1.1 is removed. Further, the NNG_TLS_1_0 and NNG_TLS_1_1 constants are also removed. Applications should use NNG_TLS_1_2 or even NNG_TLS_1_3 instead.

Only One TLS Key/Cert Per Configuration

The ability to configure multiple keys and certificates for a given TLS configuration object is removed. (The nng_tls_config_own_cert will return NNG_EBUSY if it has already been called for the configuration.) The intended purpose was to support alternative cryptographic algorithms, but this is not necessary, was never used, and was error prone.

Support for Local Addresses in Dial URLs Removed

NNG 1.x had an undocumented ability to specify the local address to bind to when dialing, by using the local address in front of the destination address separated by a semicolon. This was provided for legacy libnanomsg compatilibility, and is no longer offered. The correct way to specify a local address is by setting NNG_OPT_LOCADDR on the dialer.

IPC Option Type Changes

The types of NNG_OPT_PEER_GID, NNG_OPT_PEER_PID, NNG_OPT_PEER_UID, and NNG_OPT_PEER_ZONEID have changed from uint64_t to int. The underlying platforms all use 32-bit quantities for these.

Option Functions

The previously deprecated nng_pipe_getopt_xxx family of functions is removed. Applications should use nng_pipe_get and related functions instead.

The socket option function families for nng_getopt and nng_setopt have been removed as well. In this case, use the nng_socket_get and nng_socket_set functions as appropriate.

The _getopt and _setopt functions for contexts, listeners, and dialers are no longer present. Simply changing _getopt to _get or _setopt to _set in the function name should be sufficient in most cases.

The following functions served no useful purpose (after other changes described in this document), and are thus removed:

• nng_ctx_get_string
• nng_ctx_set_string
• nng_ctx_get_uint64
• nng_dialer_get_ptr
• nng_dialer_set_ptr
• nng_dialer_get_uint64
• nng_dialer_set_uint64
• nng_listener_get_ptr
• nng_listener_set_ptr
• nng_listener_get_uint64
• nng_listener_set_uint64
• nng_socket_get_ptr
• nng_socket_set_ptr
• nng_socket_get_string
• nng_socket_set_string
• nng_socket_get_uint64
• nng_socket_set_uint64
• nng_stream_get_ptr
• nng_stream_set_ptr
• nng_stream_get_uint64
• nng_stream_dialer_get_ptr
• nng_stream_dialer_set_ptr
• nng_stream_dialer_get_uint64
• nng_stream_dialer_set_uint64
• nng_stream_listener_get_ptr
• nng_stream_listener_set_ptr
• nng_stream_listener_get_uint64
• nng_stream_listener_set_uint64
• nng_ctx_get_ptr (not documented)
• nng_ctx_set_ptr (not documented)

Untyped Option Functions Removed

The following functions are removed. To access options, use a proper typed access function, such as one ending in a suffix like _bool (to access a bool typed option).

• nng_ctx_get
• nng_ctx_set
• nng_dialer_get
• nng_dialer_set
• nng_listener_get
• nng_listener_set
• nng_pipe_get
• nng_socket_get
• nng_socket_set
• nng_stream_get
• nng_stream_set
• nng_stream_dialer_get
• nng_stream_dialer_set
• nng_stream_listener_get
• nng_stream_listener_set

Stream Options

The ability to set options on streams after they have been created is no longer present. (It turns out that this was not very useful.) All functions nng_stream_set_xxx are removed. For tuning the NNG_OPT_TCP_NODELAY or similar properties, set the option on the listener or dialer that creates the stream instead.

Transport Options

A number of transport options can no longer be set on the socket. Instead these options must be set on the endpoint (dialer or listener) using the appropriate nng_dialer_set or nng_listener_set option. This likely means that it is necessary to allocate and configure the endpoint before attaching it to the socket. This will also afford a much more fine-grained level of control over transport options.

The following options are copied from the socket when creating a dialer or listener, but afterwards will not be changed on the dialer or listener if the socket changes. It is recommended to set them properly on the socket before creating dialers or listeners, or set them explicitly on the dialer or listener directly:

• NNG_OPT_RECONNMINT
• NNG_OPT_RECONNMAXT
• NNG_OPT_RECVMAXSZ

The latter option is a hint for transports and intended to facilitate early detection (and possibly avoidance of extra allocations) of oversize messages, before bringing them into the socket itself.

Socket Options

The NNG_OPT_PROTO, NNG_OPT_PROTONAME, NNG_OPT_PEER, and NNG_OPT_PEERNAME options have been replaced by functions instead of options. Use nng_socket_proto_id, nng_socket_peer_id, nng_socket_proto_name, and nng_socket_peer_name instead. Note that the new functions provide a reference to a static string, and thus do not require allocation, and the returned strings should not be freed. Also the IDs are provided as uint16_t, matching the actual wire protocol values, instead of int.

The NNG_OPT_RAW option has aso been replaced by a function, nng_socket_raw.

The NNG_OPT_SENDFD and NNG_OPT_RECVFD options have been replaced by nng_socket_get_send_poll_fd and nng_socket_get_recv_poll_fd respectively.

The NNG_OPT_SOCKNAME function is removed. This was provided for application use, and never used internally by NNG. Applications should keep track of this information separately.

Subscriptions

The NNG_OPT_SUB_SUBSCRIBE and NNG_OPT_SUB_UNSUBCRIBE options have been replaced by the following functions: nng_sub0_socket_subscribe, nng_sub0_socket_unsubscribe, nng_sub0_ctx_subscribe and nng_sub0_ctx_unsubscribe. These functions, like the options they replace, are only applicable to SUB sockets.

Statistics Use Constified Pointers

A number of the statistics functions take, or return, const nng_stat * instead of plain nng_stat *. The ABI has not changed, but it may be necessary to declare certain methods variables const to avoid warnings about misuse of const.

Wildcards Not Valid in URLs

The use of * to act as a wild card meaning all local interface addresses is removed. The empty string already performs this function, and unlike * is RFC compliant.

URL Option Removed

The NNG_OPT_URL option has been removed. It is replaced by the type safe nng_dialer_get_url and nng_listener_get_url functions, which return an nng_url structure instead of a string.

URL Structure Changes

The details of nng_url have changed significantly, and direct access of the structure is no longer permitted. Intead new accessors functions are provided:

• u_scheme is replaced by nng_url_scheme.
• u_port is replaced by nng_url_port, but this returns a uint16_t.
• u_hostname is replaced by nng_url_hostname.
• u_path is replaced by nng_url_path.
• u_query is replaced by nng_url_query.
• u_fragment is replaced by nng_url_fragment.
• u_userinfo is replaced by nng_url_userinfo.
• u_requri is removed - it can be easily formulated from the other fields.
• u_host is removed - use nng_url_hostname and nng_url_port to construct if needed
• u_rawurl is removed - a “cooked” URL can be obtained from the new nng_url_sprintf function.

HTTP API

The entire HTTP API has been refactored and should be much simpler to use and more efficient. Applications directly using the HTTP API will need to be fully modified.

A few limits on string lengths of certain values are now applied, which allows us to preallocate values and eliminate certain unreasonable error paths. If values longer than these are supplied in certain APIs they may be silently truncated to the limit:

• Hostnames are limited per RFC 1035 to 253 characters (not including terminating “.” or zero byte.)
• HTTP Method names are limited to 32 bytes (the longest IANA registered method is currently 18 bytes, used for WebDAV.)
• The fixed part of URI pathnames used with HTTP handlers is limited to 1024 bytes. (Longer URIs may be accepted by using [nng_http_handler_set_tree] and matching a parent of the directory component.)

The following API calls have changed so that they are void returns, and cannot fail. They may silently truncate data.

• [nng_http_req_set_method]
• [nng_http_res_set_status]
• [nng_http_handler_collect_body]
• [nng_http_handler_set_data]
• [nng_http_handler_set_host]
• [nng_http_handler_set_method]
• [nng_http_handler_set_tree]

The HTTP handler objects may not be modified once in use. Previously this would fail with NNG_EBUSY. These checks are removed now, but debug builds will assert if an application tries to do so.

WebSocket API

The NNG_OPT_WSS_REQUEST_HEADERS, NNG_OPT_WSS_RESPONSE_HEADERS and NNG_OPT_WS_OPT_WS_REQUEST_HEADERS, NNG_OPT_WS_RESPONSE_HEADERS have been removed.

The NNG_OPT_WS_REQUEST_HEADER and NNG_OPT_WS_RESPONSE_HEADER option prefixes have been collapsed into just NNG_OPT_WS_HEADER, with slightly different semantics. It still is a prefix (append the name of the header of interest), but setting it can only affect outbound headers (request header for dialers, response header for listeners), and when reading it on a pipe, the value returned is the header sent by the remote peer.

The undocumented hook function signature has changed to reflect changes in the HTTP API.

Security Descriptors (Windows Only)

The NNG_OPT_IPC_SECURITY_DESCRIPTOR option is removed, and replaced with the functions [nng_listener_get_security_descriptor] and [nng_stream_listener_get_security_descriptor].

Security descriptor support is only relevant to Windows, and is presently only supported for IPC when Named Pipes are used. Planned future changes to switch to UNIX domain sockets may eliminate support for security descriptors altogether in NNG.

Command Line Argument Parser Changes

The supplemental function nng_opts_parse and supporting definitions have moved. This functionality is now supplied by a header only library, available in nng/args.h. See nng_args_parse for more information.

ZeroTier Support Removed

The Layer 2 special ZeroTier transport has been removed. It is possible to use NNG with ZeroTier using TCP/IP, and a future update is planned to provided coexistence between ZeroTier & the native stack’s TCP/IP using lwIP.

Migrating from libnanomsg

Previous version of NNG offered direct API compatibility with libnanomsg, but that support is no longer offered in this version.

If your application is still using legacy libnanomsg APIs, you will need to update it for this version of NNG.

Header Files

Most applications can replace all #include <nn/*.h> statements with #include <nng/nng.h>.

Link Libraries

Replace -lnanomsg with -lnng. It may be necessary to include additional system libraries, such as -lpthread, depending on your system.

Initialization

It is necessary to explicitly initialize the library, using the nng_init function. This must be called before any other function.

Types

Sockets, dialers, and listeners in libnanomsg are simple integers. In NNG, these are struct types.

Messages are quite different in NNG, with the absence of the POSIX message control headers.

The struct nn_msghdr structure has no equivalent. See nng_msg for the NNG approach to messages. Likewise there is no struct nn_cmsghdr equivalent.

API Conversions

Options

The following options are changed.

Error Codes

Most of the error codes have similar names in NNG, just prefixed with NNG_. There are some exceptions. Be aware that the numeric values are not the same.

Local Addresses for Dialing

The ability to specify the source address in the URL,to use when using nn_dial inside the URL is not present in NNG. The correct way to specify the local address is using the NNG_OPT_LOCADDR option on the dialer before starting to dial.

Index

ABI, 1
abstract sockets, 1
abstract://, 1
accept, 1
AF_UNIX, 1
aio, 1
allocations, 1
application binary interface, 1
asynchronous I/O, 1
back-pressure, 1
backtrace, 1, 2, 3, 4
best-effort, 1
bind, 1
body, 1
BUS, 1
BUS protocol, 1
callback, 1
clock, 1
command-line arguments, 1
concurrency, 1
concurrent, 1
condition variable, 1
connect, 1
connection, 1
context, 1
CPU-bound, 1
duration, 1
error message, 1
event loop, 1
flow control, 1
gather, 1
getopt, 1
half-duplex, 1
header, 1
idempotent, 1
INADDR_ANY, 1
inproc, 1
inproc transport, 1
inproc://, 1
intra-process, 1
ipc, 1
ipc transport, 1
ipc://, 1
IPv4, 1
IPv6, 1
leader election, 1
listen, 1
load-balancing, 1, 2, 3
Log handlers, 1
memory, 1
messages, 1
MTU, 1
mutex, 1
named pipes, 1
NNG_AF_UNSPEC, 1
nng_aio, 1
nng_aio_abort, 1
nng_aio_alloc, 1
nng_aio_busy, 1
nng_aio_cancel, 1
nng_aio_count, 1
nng_aio_free, 1
nng_aio_get_msg, 1
nng_aio_reap, 1
nng_aio_result, 1
nng_aio_set_expire, 1
nng_aio_set_iov, 1, 2
nng_aio_set_msg, 1
nng_aio_stop, 1
nng_aio_wait, 1
nng_alloc, 1
nng_arg_spec, 1
nng_args_parse, 1
nng_bus0_open, 1
nng_bus0_open_raw, 1
nng_clock, 1
nng_ctx, 1
nng_ctx_close, 1
nng_ctx_id, 1
NNG_CTX_INITIALIZER, 1
nng_ctx_open, 1
nng_ctx_recv, 1
nng_ctx_recvmsg, 1
nng_ctx_send, 1
nng_ctx_sendmsg, 1
nng_cv, 1
nng_cv_alloc, 1
nng_cv_free, 1
nng_cv_until, 1
nng_cv_wait, 1
nng_cv_wake, 1
nng_cv_wake1, 1
nng_duration, 1
NNG_DURATION_DEFAULT, 1
NNG_DURATION_INFINITE, 1
NNG_DURATION_ZERO, 1
NNG_ECANCELED, 1
nng_fini, 1
NNG_FLAG_NONBLOCK, 1, 2
nng_free, 1
nng_http, 1
nng_http_add_header, 1
nng_http_client, 1
nng_http_client_alloc, 1
nng_http_client_connect, 1
nng_http_client_free, 1
nng_http_client_get_tls, 1
nng_http_client_set_tls, 1
nng_http_close, 1
nng_http_copy_data, 1
nng_http_del_header, 1
nng_http_get_data, 1
nng_http_get_header, 1
nng_http_get_method, 1
nng_http_get_reason, 1
nng_http_get_status, 1
nng_http_get_version, 1
nng_http_handler, 1
nng_http_handler_alloc, 1
nng_http_handler_alloc_directory, 1
nng_http_handler_alloc_file, 1
nng_http_handler_alloc_redirect, 1
nng_http_handler_alloc_static, 1
nng_http_handler_collect_body, 1
nng_http_handler_set_data, 1
nng_http_handler_set_host, 1
nng_http_handler_set_method, 1
nng_http_handler_set_tree, 1
nng_http_hijack, 1
nng_http_next_header, 1
nng_http_read, 1
nng_http_read_all, 1
nng_http_read_response, 1
nng_http_reset, 1
nng_http_set_data, 1
nng_http_set_header, 1
nng_http_set_method, 1
nng_http_set_status, 1
nng_http_set_uri, 1
nng_http_set_version, 1
nng_http_status, 1
nng_http_transact, 1, 2
nng_http_write, 1
nng_http_write_all, 1
nng_http_write_request, 1
nng_http_write_response, 1
nng_id_alloc, 1
nng_id_get, 1
nng_id_map, 1
nng_id_map_alloc, 1
nng_id_map_free, 1
nng_id_map_set, 1
nng_id_remove, 1
nng_id_visit, 1
nng_init, 1
nng_iov, 1, 2
nng_log, 1
nng_log_auth, 1
nng_log_get_level, 1
nng_log_level, 1
nng_log_set_facility, 1
nng_log_set_level, 1
nng_log_set_logger, 1
nng_logger, 1
NNG_MAJOR_VERSION, 1
NNG_MINOR_VERSION, 1
nng_msg, 1
nng_msg_alloc, 1
nng_msg_append, 1
nng_msg_body, 1
nng_msg_capacity, 1
nng_msg_chop, 1
nng_msg_clear, 1
nng_msg_dup, 1
nng_msg_free, 1
nng_msg_get_pipe, 1
nng_msg_header, 1
nng_msg_header_append, 1
nng_msg_header_chop, 1
nng_msg_header_clear, 1
nng_msg_header_insert, 1
nng_msg_header_len, 1
nng_msg_header_trim, 1
nng_msg_insert, 1
nng_msg_len, 1
nng_msg_realloc, 1
nng_msg_reserve, 1
nng_msg_set_pipe, 1
nng_msg_trim, 1
nng_msleep, 1
nng_mtx, 1
nng_mtx_free, 1
nng_mtx_lock, 1, 2
nng_mtx_unlock, 1
nng_null_logger, 1
NNG_OPT_LISTEN_FD, 1
NNG_OPT_REQ_RESENDTICK, 1
NNG_OPT_REQ_RESENDTIME, 1
NNG_OPT_SOCKET_FD, 1
NNG_OPT_SUB_PREFNEW, 1
NNG_OPT_SURVEYOR_SURVEYTIME, 1
nng_pair0_open, 1
nng_pair0_open_raw, 1
nng_pair1_open, 1
nng_pair1_open_poly, 1
nng_pair1_open_raw, 1
NNG_PATCH_VERSION, 1
nng_pipe, 1
nng_pipe_close, 1
nng_pipe_dialer, 1
NNG_PIPE_EV_ADD_POST, 1
NNG_PIPE_EV_ADD_PRE, 1
NNG_PIPE_EV_REM_POST, 1
nng_pipe_get_addr, 1
nng_pipe_get_bool, 1
nng_pipe_get_int, 1
nng_pipe_get_ms, 1
nng_pipe_get_size, 1
nng_pipe_get_string, 1
nng_pipe_id, 1
NNG_PIPE_INITIALIZER, 1
nng_pipe_listener, 1
nng_pipe_notify, 1
nng_pipe_socket, 1
nng_pub0_open, 1
nng_pub0_open_raw, 1
nng_pull0_open, 1
nng_pull0_open_raw, 1
nng_push0_open, 1
nng_push0_open_raw, 1
nng_random, 1
nng_recv, 1
nng_recvmsg, 1
nng_rep0_open, 1
nng_rep0_open_raw, 1
nng_req0_open, 1
nng_req0_open_raw, 1
nng_respondent0_open, 1
nng_respondent0_open_raw, 1
nng_send, 1
nng_sendmsg, 1
nng_sleep_aio, 1
nng_socket, 1
nng_socket_close, 1
nng_socket_get_recv_poll_fd, 1
nng_socket_get_send_poll_fd, 1
nng_socket_id, 1
nng_socket_pair, 1
nng_socket_peer_id, 1
nng_socket_peer_name, 1
nng_socket_proto_id, 1
nng_socket_proto_name, 1
nng_socket_raw, 1
nng_socket_recv, 1
nng_socket_send, 1
nng_stat, 1
nng_stat_bool, 1
NNG_STAT_BOOLEAN, 1
nng_stat_child, 1
NNG_STAT_COUNTER, 1
nng_stat_desc, 1
nng_stat_find, 1
nng_stat_find_dialer, 1
nng_stat_find_listener, 1
nng_stat_find_socket, 1
NNG_STAT_ID, 1
NNG_STAT_LEVEL, 1
nng_stat_name, 1
nng_stat_next, 1
nng_stat_parent, 1
NNG_STAT_SCOPE, 1
NNG_STAT_STRING, 1
nng_stat_string, 1
nng_stat_timestamp, 1
nng_stat_type, 1
nng_stat_unit, 1
nng_stat_value, 1
nng_stats_free, 1
nng_stats_get, 1
nng_stderr_logger, 1
nng_strdup, 1
nng_stream, 1
nng_stream_close, 1
nng_stream_dialer_alloc, 1
nng_stream_dialer_alloc_url, 1
nng_stream_dialer_close, 1
nng_stream_dialer_dial, 1
nng_stream_dialer_free, 1
nng_stream_dialer_get_addr, 1
nng_stream_dialer_get_bool, 1
nng_stream_dialer_get_int, 1
nng_stream_dialer_get_ms, 1
nng_stream_dialer_get_size, 1
nng_stream_dialer_get_string, 1
nng_stream_dialer_get_tls, 1
nng_stream_dialer_set_addr, 1
nng_stream_dialer_set_bool, 1
nng_stream_dialer_set_int, 1
nng_stream_dialer_set_ms, 1
nng_stream_dialer_set_size, 1
nng_stream_dialer_set_string, 1
nng_stream_dialer_set_tls, 1
nng_stream_dialer_stop, 1
nng_stream_free, 1
nng_stream_get_addr, 1
nng_stream_get_bool, 1
nng_stream_get_int, 1
nng_stream_get_ms, 1
nng_stream_get_size, 1
nng_stream_get_string, 1
nng_stream_listener, 1, 2
nng_stream_listener_accept, 1
nng_stream_listener_alloc, 1
nng_stream_listener_alloc_url, 1
nng_stream_listener_close, 1
nng_stream_listener_free, 1
nng_stream_listener_get_addr, 1
nng_stream_listener_get_bool, 1
nng_stream_listener_get_int, 1
nng_stream_listener_get_ms, 1
nng_stream_listener_get_size, 1
nng_stream_listener_get_string, 1
nng_stream_listener_get_tls, 1
nng_stream_listener_listen, 1
nng_stream_listener_set_addr, 1
nng_stream_listener_set_bool, 1
nng_stream_listener_set_int, 1
nng_stream_listener_set_ms, 1
nng_stream_listener_set_size, 1
nng_stream_listener_set_string, 1
nng_stream_listener_set_tls, 1
nng_stream_listener_stop, 1
nng_stream_recv, 1
nng_stream_send, 1
nng_stream_stop, 1
nng_strerror, 1
nng_strfree, 1
nng_sub0_open, 1
nng_sub0_open_raw, 1
nng_surveyor0_open, 1
nng_surveyor0_open_raw, 1
nng_system_logger, 1
nng_thread, 1
nng_thread_create, 1
nng_thread_destroy, 1
nng_thread_set_name, 1
nng_time, 1
NNG_UNIT_BYTES, 1
NNG_UNIT_EVENTS, 1
NNG_UNIT_MESSAGES, 1
NNG_UNIT_MILLIS, 1
NNG_UNIT_NONE, 1
nng_url_clone, 1
nng_url_fragment, 1
nng_url_free, 1
nng_url_hostname, 1
nng_url_parse, 1
nng_url_path, 1
nng_url_port, 1
nng_url_query, 1
nng_url_resolve_port, 1
nng_url_scheme, 1
nng_url_sprintf, 1
nng_url_userinfo, 1
nng_version, 1
nni_http_get_uri, 1
PAIR, 1
PAIR protocol, 1
pipe, 1
pipeline pattern, 1, 2, 3
polyamorous mode, 1
port number, 1
protocol, 1
PUB, 1
PUB protocol, 1
publisher, 1
PULL, 1
PULL protocol, 1
PUSH, 1
PUSH protocol, 1
random number, 1
raw mode, 1
REP, 1
REP protocol, 1
REQ, 1
REQ protocol, 1
request/reply pattern, 1, 2, 3
RESPONDENT, 1
RESPONDENT protocol, 1
scatter, 1
scheduling, 1
service discovery, 1
socket, 1
socket, 1
socket activation, 1
socket transport, 1
socket://, 1
state machines, 1
statistics, 1
stream factory, 1
streams, 1
SUB, 1
SUB protocol, 1
subscriber, 1
survey pattern, 1, 2
SURVEYOR, 1
SURVEYOR protocol, 1
synchronization primitives, 1
threads, 1
thundering herd, 1
timeout, 1
TLS, 1, 2
UDP, 1
udp transport, 1
udp://, 1
Universal Resource Locator, 1
UNIX domain sockets, 1
unix://, 1
URI, 1
URL, 1, 2
version number, 1
voting, 1