inferenceserver

The NVIDIA Inference Server provides a cloud inferencing solution optimized for NVIDIA GPUs. The server provides an inference service via an HTTP endpoint, The Inference Server itself is provided in this container. External to the container there are additional C++ and Python client libraries.

Running  NVIDIA  inferenceserver :

Before running the container, use docker pull to ensure an up-to-date image is installed. Once the pull is complete, you can run the container image.

Procedure :

1. In the Tags section, locate the container image release that you want to run.
2. In the Pull column, click the icon to copy the docker pull command.
3. Open a command prompt and paste the pull command. The pulling of the container image begins. Ensure the pull completes successfully before proceeding to the next step.
4. Run the inference server. Use nvidia-docker to run the inference server within the container.

 

For any queries, raise a ticket in the helpdesk or please contact System Administrator, #103,SERC.