How we launched our own Chatbot model with a cheap Hetzner server

Introduction

These days, launching a Dedicated Chatbot On a private server is more attractive than ever, both for economic reasons and security concerns. Large cloud services like AWS or Google Cloud are great, but their costs can be prohibitive, especially for small or test projects.
We decided on a cheap server from Hetzner Let's take a look at the experience of setting up a Chatbot. In this article, we will explain in detail what steps we took from choosing a server to the final implementation, what problems we encountered, and how we solved them.

Why did we choose Hetzner?

1. Affordable cost

Hetzner is one of the cheapest European providers in terms of price. For example, a server with 4 CPU cores and 8 GB of RAM costs around 6 euros per month It costs money.

2. Quality and sustainability

Hetzner's German and Finnish data centers are very stable and fast, and have acceptable latency even for Iranian user traffic.

3. Hardware diversity

From small VPS to dedicated servers with powerful GPUs, you can choose according to your needs and budget.

Steps to set up a Chatbot on Hetzner

Step 1: Choose an operating system

We installed Ubuntu 22.04 LTS. This version is both lightweight and has the most support for AI-related tools.

Step 2: Install basic tools

After logging into the server, you need to install Python and the required tools:

sudo apt update && sudo apt upgrade -y
sudo apt install python3-pip git -y
pip install transformers accelerate fastapi uvicorn

from transformers import AutoModelForCausalLM, AutoTokenizer
from fastapi import FastAPI

app = FastAPI()

tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-medium")
model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-medium")

@app.get("/chat")
def chat(msg: str):
inputs = tokenizer.encode(msg + tokenizer.eos_token, return_tensors="pt")
outputs = model.generate(inputs, max_length=1000, pad_token_id=tokenizer.eos_token_id)
reply = tokenizer.decode(outputs[:, inputs.shape[-1]:][0], skip_special_tokens=True)
return {"reply": reply}

uvicorn main:app --host 0.0.0.0 --port 8000

curl "http://your-server-ip:8000/chat?msg=سلام"