In this ultra-light Mistral Devstral Tutorial, a friendly colaab guide is provided which is specifically designed for users faced with disk space constraints. The execution of large languages like Mistral can be a challenge in environments with limited storage and memory, but this tutorial shows how to deploy the powerful Devstral-Small model. With aggressive quantification using bitsandbytes, cache management and a generation of effective tokens, this tutorial guides you through the construction of a light assistant which is fast, interactive and concerned with disc. Whether you are debit in code, you write small tools or prototying on the move, this configuration guarantees that you get maximum performance with a minimum imprint.
!pip install -q kagglehub mistral-common bitsandbytes transformers --no-cache-dir
!pip install -q accelerate torch --no-cache-dir
import shutil
import os
import gcThe tutorial begins by installing essential light packages such as KaggleHub, Mistral-Common, Bitsandbytes and Transformers, ensuring that no cache is stored to minimize the use of the disc. It also includes acceleration and torch for loading and inference of effective models. To further optimize the space, any pre -existing or temporary directory cover is erased using the Shutil, OS and GC modules.
def cleanup_cache():
"""Clean up unnecessary files to save disk space"""
cache_dirs = ('/root/.cache', '/tmp/kagglehub')
for cache_dir in cache_dirs:
if os.path.exists(cache_dir):
shutil.rmtree(cache_dir, ignore_errors=True)
gc.collect()
cleanup_cache()
print("🧹 Disk space optimized!")To maintain a minimum disk footprint throughout the execution, the cleanup_cache () function is defined to remove redundant cache directories like /root/.cache and / TMP / KAGGLEHUB. This proactive cleaning helps to release the space before and after key operations. Once invoked, the function confirms that the disk space has been optimized, strengthening the accent of the tutorial on the effectiveness of resources.
import warnings
warnings.filterwarnings("ignore")
import torch
import kagglehub
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from transformers import AutoModelForCausalLM, BitsAndBytesConfigTo ensure fluid execution without distracting warning messages, we delete all execution warnings using the Python warning module. It is then important essential libraries for the interaction of the model, including the torch for the tensor's calculations, the KaggleHub for the streaming of the model and the transformers for the loading of the quantified LLM. Specific classes in Mistral such as Usermessage, Catcompletionrequest and Mistraltokenzer are also excited to manage tokenization and request formatting suitable for Devstral's architecture.
class LightweightDevstral:
def __init__(self):
print("📦 Downloading model (streaming mode)...")
self.model_path = kagglehub.model_download(
'mistral-ai/devstral-small-2505/Transformers/devstral-small-2505/1',
force_download=False
)
quantization_config = BitsAndBytesConfig(
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_storage=torch.uint8,
load_in_4bit=True
)
print("⚡ Loading ultra-compressed model...")
self.model = AutoModelForCausalLM.from_pretrained(
self.model_path,
torch_dtype=torch.float16,
device_map="auto",
quantization_config=quantization_config,
low_cpu_mem_usage=True,
trust_remote_code=True
)
self.tokenizer = MistralTokenizer.from_file(f'{self.model_path}/tekken.json')
cleanup_cache()
print("✅ Lightweight assistant ready! (~2GB disk usage)")
def generate(self, prompt, max_tokens=400):
"""Memory-efficient generation"""
tokenized = self.tokenizer.encode_chat_completion(
ChatCompletionRequest(messages=(UserMessage(content=prompt)))
)
input_ids = torch.tensor((tokenized.tokens))
if torch.cuda.is_available():
input_ids = input_ids.to(self.model.device)
with torch.inference_mode():
output = self.model.generate(
input_ids=input_ids,
max_new_tokens=max_tokens,
temperature=0.6,
top_p=0.85,
do_sample=True,
pad_token_id=self.tokenizer.eos_token_id,
use_cache=True
)(0)
del input_ids
torch.cuda.empty_cache() if torch.cuda.is_available() else None
return self.tokenizer.decode(output(len(tokenized.tokens):))
print("🚀 Initializing lightweight AI assistant...")
assistant = LightweightDevstral()We define the Lightweightdevstral class, the central component of the tutorial, which manages the loading of the model and the generation of text in a resource manner. It starts by broadcasting the Devstral-Small-2505 model using KaggleHub, avoiding redundant downloads. The model is then in charge of an aggressive 4 -bit quantification via bitsandbytesconfig, considerably reducing the use of memory and disc while allowing performance inference. A personalized token is initialized from a local json file and the cache is erased immediately after. The generation method uses memory safety practices, such as Torch.inference_Mode () and Vide_cache (), to generate responses effectively, which makes this assistant adapted even to environments with narrow material constraints.
def run_demo(title, prompt, emoji="🎯"):
"""Run a single demo with cleanup"""
print(f"\n{emoji} {title}")
print("-" * 50)
result = assistant.generate(prompt, max_tokens=350)
print(result)
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
run_demo(
"Quick Prime Finder",
"Write a fast prime checker function `is_prime(n)` with explanation and test cases.",
"🔢"
)
run_demo(
"Debug This Code",
"""Fix this buggy function and explain the issues:
```python
def avg_positive(numbers):
total = sum((n for n in numbers if n > 0))
return total / len((n for n in numbers if n > 0))
```""",
"🐛"
)
run_demo(
"Text Tool Creator",
"Create a simple `TextAnalyzer` class with word count, char count, and palindrome check methods.",
"🛠️"
)
Here, we present the coding capacities of the model via a compact demonstration series using the RUN_DEMO () function. Each demo sends an prompt to the Devstral assistant and prints the response generated, immediately followed by memory cleaning to prevent accumulation on several analyzes. The examples include writing a first-rate effective verification function, debugging a python extract with logical defects and the construction of a Textanalyzer mini-class. These demonstrations highlight the usefulness of the model as a light coding assistant and concerned with the disc capable of generation and explanation of code in real time.
def quick_coding():
"""Lightweight interactive session"""
print("\n🎮 QUICK CODING MODE")
print("=" * 40)
print("Enter short coding prompts (type 'exit' to quit)")
session_count = 0
max_sessions = 5
while session_count < max_sessions:
prompt = input(f"\n({session_count+1}/{max_sessions}) Your prompt: ")
if prompt.lower() in ('exit', 'quit', ''):
break
try:
result = assistant.generate(prompt, max_tokens=300)
print("💡 Solution:")
print(result(:500))
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
except Exception as e:
print(f"❌ Error: {str(e)(:100)}...")
session_count += 1
print(f"\n✅ Session complete! Memory cleaned.")We introduce the quick coding mode, a light interactive interface that allows users to submit short coding prompts directly to the Devstral assistant. Designed to limit the use of memory, the interaction of the session caps at five prompts, each followed by aggressive memory cleaning to ensure continuous responsiveness in low -resource environments. The assistant responds with concise and truncated code suggestions, which makes this ideal mode for rapid prototyping, debugging or exploring coding concepts on the fly, all without crushing the disc or the memory capacity of the notebook.
def check_disk_usage():
"""Monitor disk usage"""
import subprocess
try:
result = subprocess.run(('df', '-h', '/'), capture_output=True, text=True)
lines = result.stdout.split('\n')
if len(lines) > 1:
usage_line = lines(1).split()
used = usage_line(2)
available = usage_line(3)
print(f"💾 Disk: {used} used, {available} available")
except:
print("💾 Disk usage check unavailable")
print("\n🎉 Tutorial Complete!")
cleanup_cache()
check_disk_usage()
print("\n💡 Space-Saving Tips:")
print("• Model uses ~2GB vs original ~7GB+")
print("• Automatic cache cleanup after each use")
print("• Limited token generation to save memory")
print("• Use 'del assistant' when done to free ~2GB")
print("• Restart runtime if memory issues persist")Finally, we offer a cleaning routine and a user -use monitor. Using the DF -H command via the Python sub -process module, it displays the amount of disk space used and available, confirming the light nature of the model. After having reinstated cleanup_cache () to ensure a minimum of residues, the script ends with a set of practical space saving advice.
In conclusion, we can now take advantage of the capacities of the Mistral Devstral model in limited environments in space such as Google Colab, without compromising usability or speed. The model is responsible in a highly compressed format, performs an effective text generation and guarantees that memory is quickly erased after use. With the interactive coding mode and the sequence of demonstration included, users can test their ideas quickly and transparently.
Discover the Codes. All the merit of this research goes to researchers in this project. Also, don't hesitate to follow us Twitter And don't forget to join our Subseubdredit 100k + ml and subscribe to Our newsletter.
Asif Razzaq is the CEO of Marktechpost Media Inc .. as a visionary entrepreneur and engineer, AIF undertakes to exploit the potential of artificial intelligence for social good. His most recent company is the launch of an artificial intelligence media platform, Marktechpost, which stands out from its in-depth coverage of automatic learning and in-depth learning news which are both technically solid and easily understandable by a large audience. The platform has more than 2 million monthly views, illustrating its popularity with the public.
