py-smi

14 of december of 2024

py-smi

Disclaimer: This post has been translated to English using a machine translation model. Please, let me know if you find any mistakes.

Do you want to be able to use nvidia-smi from Python? Here you have a library to do it.

Installation

To install it, run:

pip install python-smi```

Usage

We import the library

	
		from py_smi import NVML

We create an object of pynvml (the library behind nvidia-smi)

	
		nv = NVML()

We get the version of the driver and CUDA

	
		nv.driver_version, nv.cuda_version

	
		('560.35.03', '12.6')

Since in my case I have two GPUs, I create a variable with the number of GPUs

	
		num_gpus = 2

I get the memory of each GPU

	
		[nv.mem(i) for i in range(num_gpus)]

	
		[_Memory(free=24136.6875, total=24576.0, used=439.3125),
 _Memory(free=23509.0, total=24576.0, used=1067.0)]

The utilization

	
		[nv.utilization() for i in range(num_gpus)]

	
		[_Utilization(gpu=0, memory=0, enc=0, dec=0),
 _Utilization(gpu=0, memory=0, enc=0, dec=0)]

The power used

This works very well for me, because when I was training a model and had both GPUs full, sometimes my computer would shut down, and looking at the power, I see that the second one consumes a lot, so it may be what I already suspected, which was due to power supply.

	
		[nv.power(i) for i in range(num_gpus)]

	
		[_Power(usage=15.382, limit=350.0), _Power(usage=40.573, limit=350.0)]

The clocks of each GPU

	
		[nv.clocks(i) for i in range(num_gpus)]

	
		[_Clocks(graphics=0, sm=0, mem=405), _Clocks(graphics=540, sm=540, mem=810)]

PCI Data

	
		[nv.pcie_throughput(i) for i in range(num_gpus)]

	
		[_PCIeThroughput(rx=0.0, tx=0.0),
 _PCIeThroughput(rx=0.1630859375, tx=0.0234375)]

And the processes (I'm not running anything now)

	
		[nv.processes(i) for i in range(num_gpus)]

	
		[[], []]

Continue reading

RAG fundamentals

Forget about Ctrl+F! 🤯 With RAG, your documents will answer your questions directly. 😎 Step-by-step tutorial with Hugging Face and ChromaDB. Unleash the power of AI (and show off to your friends)! 💪

Conventional commits

😠 Are your commits written in alien language? 👽 Join the club! 😅 Learn Conventional Commits in Python and stop torturing your team with cryptic messages. git-changelog and commitizen will be your new best friends. 🤝

DoLa – Decoding by Contrasting Layers Improves Factuality in Large Language Models

Have you ever talked to an LLM and they answered you something that sounds like they've been drinking machine coffee all night long 😂 That's what we call a hallucination in the LLM world! But don't worry, because it's not that your language model is crazy (although it can sometimes seem that way 🤪). The truth is that LLMs can be a bit... creative when it comes to generating text. But thanks to DoLa, a method that uses contrast layers to improve the feasibility of LLMs, we can keep our language models from turning into science fiction writers 😂. In this post, I'll explain how DoLa works and show you a code example so you can better understand how to make your LLMs more reliable and less prone to making up stories. Let's save our LLMs from insanity and make them more useful! 🚀

Last posts -->

Have you seen these projects?

Subtify

GPU monitor

Alfred

View all projects -->

Do you want to apply AI in your project? Contact me!

Do you want to improve with these tips?

DataLoader with pin_memory and num_workers

Increase DataLoader performance with pin_memory and num_workers

Memory Calculator

Space to calculate the memory needed to run a model

Code formatter Black

Your colleague Patric is writing code that is hard to read? Share with him this code formatter that I show you in this post! Come in and learn how to format code to make it more understandable. We are not going to solve Patric's problems, but at least you won't suffer when reading it

Last tips -->

Use this locally

Hugging Face spaces allow us to run models with very simple demos, but what if the demo breaks? Or if the user deletes it? That's why I've created docker containers with some interesting spaces, to be able to use them locally, whatever happens. In fact, if you click on any project view button, it may take you to a space that doesn't work.