No, the Docker container does not launch with the --privileged flag
The platform manages the device states and usage through the orchestration of docker containers. We require to mount the docker socket for managing docker containers on the worker node. This is mandatory for the platform and there are currently no plans or alternatives to remove this
while the --privileged flag gives broad system access to a container, mounting the Docker socket gives the container control over Docker on the host
Our platform enables clustering gpu compute and provides the end user of the platform with a production ready environment to do distributed training. The custom docker images contain all the required drivers and environment with all libraries installed which enables efficient utilization of GPU and CPU resources which are mandatory for distributed training. Reproduction of the same environment as ours from a worker stand point is very troublesome and can have irregularities based on the platform the worker has (linux, windows), the most important requirement for distributed training is to exactly replicate the environment on all nodes.
Updated 27 days ago