Monthly report: November 2018

Product

Platform

  • Implemented a Module for the automatic management of deals and tasks — Taskman (a new bot that enables automatic resource provision on the Sonm platform). More information is available here. Documentation is available here.
  • Testing of the Sonm OS public beta with an automated installer continues. A version of Sonm OS has been created with another installation option based on the Debian installer and with a mode for booting into a “live USB” with a Windows-based software installer for to prepare the Sonm OS USB drive, which is now being tested by developers. In the near future, this version will also be released in public beta.
  • Research on the QUIC protocol was completed, and experimental support for the protocol on the SONM platform was implemented. QUIC is an experimental transport layer network protocol that we use as an alternative to TCP to achieve reliable communication between our components. The main reason we’re including it is to increase connection, establishing stability between any two peers located in private networks. QUIC is designed on top of UDP, which is more predictable and has been investigated for NAT penetration, unlike TCP. Since QUIC allows for many connections in a single socket, there is no need to penetrate NAT for the same client-server pair multiple times. After the first successful attempt, the punched hole will be reused.
  • The Rendezvous server has been updated to support UDP hole-punching using the QUIC protocol. Additionally, it exposes its gRPC server to the same UDP port as for TCP, since it is possible to reuse ports when different network protocols are used. The wire level remains the same — gRPC.
  • The function “Geolocation for Suppliers/Workers” is implemented on the side of the SONM Worker component. Now Workers can detect the country ISO code of their location and provide it in their status handler.

Changelog

v0.4.15

  • Added the ability to automatically determine the geolocation of Worker.
  • Added the ability to collect metrics from DRI devices. Now we’re trying to find the hwmon (hardware monitoring) path on the device initialization. If we can do this, the monitoring interface for this card will become available. The «Monitoring» method reads data from the “/sys/dev” internals, and all behavior has been imported from the ROC-smi tool.
  • Added the ability to collect extended network metrics in Worker. The extended network metrics, including traffic/packet minute-averaged rates (required for determining the network pressure caused by a task), are now returned when requesting container/task stats..
  • Added the ability to SSH on a Worker host. Now it is possible to SSH to a Worker using a Node proxy. Internally it works the same way as with SSH into containers, except that the user identity can be the public ETH address of a Worker.
  • General platform improvements:
  • added the function PayoutTargeted to Gatekeeper (sidechain only; the masterchain gate does not need this. PayoutTargeted creates a transfer to the target destination).
  • added used bytes to the disk benchmark (needed to avoid two syscalls to gather disk space and to monitor Workers).
  • Optimus algorithm improvements: Optimus will no longer remove active plans. The new logic in the Worker should eliminate races — all running bots are engaged in a new, profitable order, but only one of them succeeds. Now it will cancel ask-plans only if there is no deal associated with them, delegating other plan management logic to the Worker.
  • Fixed platform component bugs.
  • Added the ability to collect power consumption for GPUs.
  • Fixed platform component bugs.
  • Experimental QUIC protocol support. Affected components: Worker, Node, Rendezvous. Both Node and Worker now expose their gRPC services on both TCP and UDP sockets (the port number is the same). Note that this feature is currently in an experimental stage.
  • Added version checking for core components. Currently, we often run into a situation where our clients (both suppliers and customers) don’t know whether a new version of the platform has been released. To remedy this, we’ve added a simple console notification to let clients know when it’s time to update.
  • Added the ability to collect background metrics for Worker:
  • implemented the MetricsHandler plugin, which is bound to an instance of Worker. It’s responsible for handling hardware-specific metrics providers (currently for GPUs only) and updating the whole set of metrics by the timer, returning them to the last known state.
  • implemented the GRPC method for WorkerManagement API, which can be used to retrieve metrics using the master or admin address.
  • Rendezvous improvements:
  • implemented a replacement TCP KA with HTTP2 pings to Rendezvous. Earlier, we tried to set up a TCP keepalive option to ensure that even if the NAT closes the connection, we’re still able to detect it properly; however, the library we use to configure the SO_REUSEPORT option just ignores any keepalive specifications. This results in a situation where sometimes during either network instability or NAT conntrack configurations, the Rendezvous server properly detects whether a Worker is disconnected, but the Worker itself does not detect it; otherwise, it would try to reconnect. To avoid such cases, we have implemented a replacement TCP KA with HTTP2 pings.
  • implemented a solution to eliminate deadlock in the Rendezvous server.
  • General platform improvements:
  • added the metrics collection addr and ACL for Worker.
  • added the ability to show deal type in deal list cmd (#1749).
  • added the “type” field to the “sonmcli deal list” command. The type indicates whether the current user is selling or buying resources in the given deal.
  • added the ability to announce relay concurrently. At the moment, we have a time window where, after accepting a relayed connection, clients are forced to wait until the server publishes itself again. This window can be reduced if clients announce several connections concurrently (by default, the concurrency is set to 2, but it can be increased up to 4 in the config).
  • added a timeout in Optimus when removing plans.
  • added the ability to show master and Worker IDs in the deal status.
  • Fixed platform component bugs and implemented small improvements in system stability.
  • Implemented small improvements in system stability.
  • Improved QUIC protocol support in the system.
  • Fixed platform component bugs.

Implementations in progress:

  • There is active development and testing of the Ratings and SLA system (a system to enforce the reliability of rented resources based on the economic incentive of receiving higher ratings). The Ratings and SLA system affects many components of SONM. We have implemented the work prototype of the Ratings and SLA system. The active phase of testing the prototype will begin shortly.
  • Work is in progress on a UI for the Blockchain Explorer.
  • Work is in progress to add support for the feature “Geolocation for Suppliers/Workers” in the SONM Market.
  • Work is in progress on the prototype implementation of TensorFlow on SONM.
  • An auto-withdrawal feature is underway and being testing. It will allow hardware suppliers to specify an Ethereum address for automatic withdrawal, similar to how it works in the mining pool.

Research in progress:

  • Research is underway on deploying a CGI rendering farm on Sonm using the Afanasy render farm manager. We have successfully conducted the first tests of prototypes of Workers for video-rendering based on the SONM platform
  • Research is underway on the choice of options for a referral system to reward network users who contribute to the platform’s growth.
  • Research is underway on Market Explorer, a system to give Sonm users the ability to track deal histories and view platform usage statistics.
  • Research is underway on a system for purchasing Sonm tokens with wire transfers for corporate clients (fiat gate).
  • Research is underway on various options for implementing Gate Masternodes and Sidechain PoS to select the best solution. We are investigating Plasma options and other possible solutions.

Business & Marketing

Sonm OS

Q&A New Format

Brand Advocacy

Events

Highload ++ 2018

GoTech

CryptoEvent

Data Science Conference

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
SONM

SONM

Global Fog Computing Platform