Systems Communication

[anitsh]

Systems Communication

When we think of it as natural thing we do in our daily life, we can better paint how the systems communicate as we build system that communicates to fuction.

Data Exchange

Data exchange is the process of taking data structured under a source schema and transforming it into a target schema, so that the target data is an accurate representation of the source data. Data exchange allows data to be shared between different computer programs.

Partial list of popular generic languages used for data exchange in multiple domains.

Protocol	Schemas	Flexible	Semantic verification	Dictionary	Information Model	Synonyms and homonyms	Dialecting	Web standard	Transformations	Lightweight	Human readable	Compatibility
RDF	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Partial	Subset of Semantic web
REBOL	Yes	Yes	No	Yes	No	Yes	Yes	No	Yes	Yes	Yes
Gellish	Yes	Yes	Yes	Yes	No	Yes	Yes	ISO	No	Yes	Partial	SQL, RDF/XML, OWL
Atom	Yes	Un­known	Un­known	Un­known	No	Un­known	Yes	Yes	Yes	No	No	XML dialect
JSON	No	Un­known	Un­known	Un­known	No	Un­known	No	Yes	No	Yes	Yes	subset of YAML
YAML	No	Un­known	Un­known	Un­known	No	Un­known	No	No	No	Yes	Yes	superset of JSON
XML	Yes	Yes	No	No	No	No	Yes	Yes	Yes	No	Yes	subset of SGML, HTML

Nomenclature

• Schemas - Whether the language definition is available in a computer interpretable form
• Flexible - Whether the language enables extension of the semantic expression capabilities without modifying the schema
• Semantic verification - Whether the language definition enables semantic verification of the correctness of expressions in the language
• Dictionary-Taxonomy - Whether the language includes a dictionary and a taxonomy (subtype-supertype hierarchy) of concepts with inheritance
• Synonyms and homonyms - Whether the language includes and supports the use of synonyms and homonyms in the expressions
• Dialecting - Whether the language definition is available in multiple natural languages or dialects
• Web or ISO standard - Organization that endorsed the language as a standard
• Transformations - Whether the language includes a translation to other standards
• Lightweight - Whether a lightweight version is available, in addition to a full version
• Human-readable - Whether expressions in the language are human-readable (readable by humans without training)
• Compatibility - Which other tools are possible to use or required when using the language

Resource

• [ ] https://en.wikipedia.org/wiki/Data_exchange
• [ ] https://www.softprayog.in/programming/interprocess-communication-using-posix-message-queues-in-linux | https://www.softprayog.in/programming/interprocess-communication-using-pipes-in-linux | https://www.softprayog.in/programming/interprocess-communication-using-fifos-in-linux | https://www.softprayog.in/programming/interprocess-communication-using-system-v-message-queues-in-linux | https://www.softprayog.in/programming/interprocess-communication-using-system-v-shared-memory-in-linux | https://www.softprayog.in/programming/d-bus-tutorial

[anitsh]

POSIX

Standards are what make my computer able to communicate with yours, they make websites render the same on our respective computers, they allow our computers to interpret a live video feed do that we all see the same thing. But standard extend beyond the sharing of content between two computers, some standards define specific patterns that allow much more than network or file interoperability. POSIX is one such standard.

Linux is known to support most of the POSIX system call interfaces as well as a large extension to the POSIX specification, known as the Linux Standard Base, which aims to unify Linux distributions in both code and binary support. https://pubs.opengroup.org/onlinepubs/9699919799/idx/functions.html

Resource

• [ ] https://fossbytes.com/posix-what-is-the-portable-operating-system-interface

[anitsh]

Inter-Process Communication (IPC)

In computer science, inter-process communication or interprocess communication (IPC) refers specifically to the mechanisms an operating system provides to allow the processes to manage shared data. Typically, applications can use IPC, categorized as clients and servers, where the client requests data and the server responds to client requests. Many applications are both clients and servers, as commonly seen in distributed computing.

IPC is very important to the design process for microkernels and nanokernels, which reduce the number of functionalities provided by the kernel. Those functionalities are then obtained by communicating with servers via IPC, leading to a large increase in communication when compared to a regular monolithic kernel. IPC interfaces generally encompass variable analytic framework structures. These processes ensure compatibility between the multi-vector protocols upon which IPC models rely.

An IPC mechanism is either synchronous or asynchronous. Synchronization primitives may be used to have synchronous behavior with an asynchronous IPC mechanism.

Approaches

Different approaches to IPC have been tailored to different software requirements, such as performance, modularity, and system circumstances such as network bandwidth and latency.

Method	Short Description	Provided by (operating systems or other environments)
File	A record stored on disk, or a record synthesized on demand by a file server, which can be accessed by multiple processes.	Most operating systems
Communications file	A unique form of IPC in the late-1960s that most closely resembles Plan 9's 9P protocol	Dartmouth Time-Sharing System
Signal; also Asynchronous System Trap	A system message sent from one process to another, not usually used to transfer data but instead used to remotely command the partnered process.	Most operating systems
Socket	Data sent over a network interface, either to a different process on the same computer or to another computer on the network. Stream-oriented (TCP; data written through a socket requires formatting to preserve message boundaries) or more rarely message-oriented (UDP, SCTP).	Most operating systems
Unix domain socket	Similar to an internet socket, but all communication occurs within the kernel. Domain sockets use the file system as their address space. Processes reference a domain socket as an inode, and multiple processes can communicate with one socket	All POSIX operating systems and Windows 10[3]
Message queue	A data stream similar to a socket, but which usually preserves message boundaries. Typically implemented by the operating system, they allow multiple processes to read and write to the message queue without being directly connected to each other.	Most operating systems
Anonymous pipe	A unidirectional data channel using standard input and output. Data written to the write-end of the pipe is buffered by the operating system until it is read from the read-end of the pipe. Two-way communication between processes can be achieved by using two pipes in opposite "directions".	All POSIX systems, Windows
Named pipe	A pipe that is treated like a file. Instead of using standard input and output as with an anonymous pipe, processes write to and read from a named pipe, as if it were a regular file.	All POSIX systems, Windows, AmigaOS 2.0+
Shared memory	Multiple processes are given access to the same block of memory, which creates a shared buffer for the processes to communicate with each other.	All POSIX systems, Windows
Message passing	Allows multiple programs to communicate using message queues and/or non-OS managed channels. Commonly used in concurrency models.	Used in RPC, RMI, and MPI paradigms, Java RMI, CORBA, DDS, MSMQ, MailSlots, QNX, others
Memory-mapped file	A file mapped to RAM and can be modified by changing memory addresses directly instead of outputting to a stream. This shares the same benefits as a standard file.	All POSIX systems, Windows

Applications

Remote procedure call interfaces

Java's Remote Method Invocation (RMI)
ONC RPC
XML-RPC or SOAP
JSON-RPC
Message Bus (Mbus) (specified in RFC 3259)
.NET Remoting
gRPC

Platform communication stack

The following are messaging, and information systems that utilize IPC mechanisms but don't implement IPC themselves: KDE's Desktop Communications Protocol (DCOP) – deprecated by D-Bus D-Bus OpenWrt uses ubus micro bus architecture MCAPI Multicore Communications API SIMPL The Synchronous Interprocess Messaging Project for Linux (SIMPL) 9P (Plan 9 Filesystem Protocol) Distributed Computing Environment (DCE) Thrift ZeroC's Internet Communications Engine (ICE) ØMQ Enduro/X Middleware YAMI4

Operating system communication stack

The following are platform or programming language-specific APIs: Linux Transparent Inter Process Communication (TIPC) Apple Computer's Apple events, previously known as Interapplication Communications (IAC) Enea's LINX for Linux (open source) and various DSP and general-purpose processors under OSE The Mach kernel's Mach Ports Microsoft's ActiveX, Component Object Model (COM), Microsoft Transaction Server (COM+), Distributed Component Object Model (DCOM), Dynamic Data Exchange (DDE), Object Linking and Embedding (OLE), anonymous pipes, named pipes, Local Procedure Call, MailSlots, Message loop, MSRPC, .NET Remoting, and Windows Communication Foundation (WCF) Novell's SPX POSIX mmap, message queues, semaphores,[4] and shared memory RISC OS's messages Solaris Doors System V's message queues, semaphores, and shared memory OpenBinder Open binder QNX's PPS (Persistent Publish/Subscribe) service

Distributed object models

The following are platform or programming language specific-APIs that use IPC, but do not themselves implement it: Libt2n for C++ under Linux only, handles complex objects and exceptions PHP's sessions Distributed Ruby Common Object Request Broker Architecture (CORBA)

• Source: https://en.wikipedia.org/wiki/Inter-process_communication

Transparent Inter Process Communication service ( TIPC )

Transparent Inter Process Communication (TIPC) is an Inter-process communication (IPC) service in Linux designed for cluster wide operation. It is sometimes presented as Cluster Domain Sockets, in contrast to the well-known Unix Domain Socket service; the latter working only on a single kernel.

• [ ] https://www.ibm.com/support/pages/interprocess-communication-ipc-overview
• [ ] https://en.wikipedia.org/wiki/Transparent_Inter-process_Communication
• [ ] http://www.tipc.io/intro.html

[anitsh]

mmap

"In computing, mmap is a POSIX-compliant Unix system call that maps files or devices into memory. It is a method of memory-mapped file I/O. It implements demand paging because file contents are not read from disk directly and initially do not use physical RAM at all. The actual reads from disk are performed in a "lazy" manner, after a specific location is accessed. After the memory is no longer needed, it is important to munmap the pointers to it. Protection information can be managed using mprotect(2), and special treatment can be enforced using madvise.

In Linux, macOS and the BSDs, mmap can create several types of mappings. Other operating systems may only support a subset of these; for example, shared mappings may not be practical in an operating system without a global VFS or I/O cache. " - https://en.wikipedia.org/wiki/Mmap

"POSIX environments provide at least two ways of accessing files. There's the standard system calls open(), read(), write(), and friends, but there's also the option of using mmap() to map the file into virtual memory. When is it preferable to use one over the other? What're their individual advantages that merit including two interfaces?"

• [ ] https://stackoverflow.com/questions/258091/when-should-i-use-mmap-for-file-access

• [ ] https://brunocalza.me/discovering-and-exploring-mmap-using-go | https://brunocalza.me/discovering-and-exploring-mmap-using-go In-depth exploration of mmap started from this blog.

• [ ] https://www.xml.com/ldd/chapter/book/ch13.html - mmap and DMA

• [ ] https://medium.com/@arpith/adventures-with-mmap-463b33405223

• [ ] http://learnmongodbthehardway.com/schema/mmap/

• [ ] https://www.cs.uaf.edu/2016/fall/cs301/lecture/11_02_mmap.html

• [ ] https://news.ycombinator.com/item?id=19805675

• [ ] https://sqlite.org/mmap.html

• [ ] https://github.com/yabeda-rb/yabeda-prometheus-mmap

• [ ] https://www.kernel.org/doc/html/v4.9/media/uapi/v4l/func-mmap.html

• [ ] https://bugs.php.net/bug.php?id=77930

• [ ] https://blog.minhazav.dev/memory-sharing-in-linux/

• [ ] https://www.oreilly.com/library/view/mastering-embedded-linux/9781787283282/5d6307d0-4ca5-424a-8ff7-9fe9c56461f8.xhtml