Ollama and DeepSeek with EWB
by Erick Engelke
January 26, 2025
1. install Ollama
You will need to install Ollama first. download web site
and follow the instructions.
2.Download Deepseek R1
Then you download and run DeepSeek. There are a couple of variants, I used this:
ollama run deepseek-r1:7b3. Test DeepSeek
>>> simplify "it's time for you kids to go to bed now, you need your sleep"And you will get a response, ending with something like: "Time for the kids to go to bed."
Ask "what is the root of eveil" and see something like:
… In conclusion, evil is a complex issue resulting from both internal and external factors. Addressing it requires self-awareness, personal improvement, and a supportive environment.
Exit the program with
>>> /byeYou can see which models are available now, each has different characteristics. Use the following command to see what is downloaded.
ollama list
NAME ID SIZE MODIFIED
deepseek-r1:latest 0a8c26691023 4.7 GB 18 minutes ago
llama3.2:latest a80c4f17acd5 2.0 GB 4 hours agoUsing CURL to send a network based request (localhost only)
If you have CURL installed, you can use it to generate network based requests, though only on the local host.
curl http://localhost:11434/api/generate -d '{
"model":"deepseek-r1:7b",
"prompt":"what is an apple?"
}'Note, it sends one token (like a word) at a time, each in its own JSON structure.
EWB based Querying Tool
:
We will do the same in an EWB program…
|
Note
|
this EWB program must be run on the same device as Ollama is running, otherwise it with get a CORS error, because Ollama does not allow requests from other devices. |
The user will enter a query into MultiLineEdit1’s text box, and we submit it to the server.
procedure TForm1.btQueryClick(Sender: TObject);
begin
ServerRequest1.URL := 'http://localhost:11434/api/generate';
ServerRequest1.method := rmPost;
ServerRequest1.Content.Text := '{"model":"deepseek-r1:latest", "prompt":"'+Trim(MultilineEdit1.Lines.Text)+'"}';
ServerRequest1.OnProgress := Progress;
ServerRequest1.OnCOmplete := Complete;
ServerRequest1.OnError := Error;
ServerRequest1.Execute;
Timer1.Enabled := True;
end;Now, we could wait for it to complete, but that could take some time. So instead we set off a TTimer (Timer1) to read any updates on the HTTP session.
procedure TForm1.Timer1Timer(Sender: TObject);
begin
if assigned( TServerRequest2(ServerRequest1).request ) then
progress(0, 100 );
end;What the bleep is TServerRequest2? Well it is a cracker class which exposes EWB’s underlying HTTP request. See my post on cracker classes for more details. The Request is only valid if the HTTP request has been made, so we check to see if it’s validly assigned.
That updates TMultiLineEdit2, the response window, with any data received so far.
procedure TForm1.Progress( current : integer; total : integer );
var
req2 : TServerRequest2;
begin
req2 := TServerRequest2(ServerRequest1);
MultiLineEdit2.Lines.Text := 'update:' + getAllWords( req2.request.ResponseText);
end;The JSON is represented in EWB as follows:[source,]
type
external TResults = class( TExternalObject )
public
property model : string read write;
property response : string read write;
property done :boolean read write;
end;
GetAllWords returns the JSON messages converted into individual tokens (numbers, words, icons, etc.).
function TForm1.GetAllWOrds( inp : string ): string;
var
i , len : integer;
bigs, s : string;
struc : TResults;
begin
result := '';
repeat
i := pos('}',inp ) ;
s := copy( inp, 1, i );
inp := Copy( inp, i+1 );
if s = '' then break;
try
struc := TResults( JSONParse( s ) );
except on E:Exception do
result := E.message + ' "' + s +'"';
end;
try
result := result + struc.response;
except
end;
until inp = '';
end;JSONParse is library function in my EWB Nice toolkit.
And Complete is called with the complete data is received.
procedure TForm1.Complete( req : TServerRequest );
begin
//
Timer1.Enabled := false;
MultiLineEdit2.Lines.Text := 'Complete: ' + getAllWords( req.responsecontent.Text ) ;
end;