From creative writing to professional content creation, KoboldAI is one of the great solution and an alternative of OpenAI for AI-assisted writing It also provides a seamless and intuitive experience that elevates your writing process.
It has a browser-based front-end that allows users to create and edit stories, novels, chatbots, and more with the help of tools such as Memory, Author’s Note, World Info, Save & Load, and other adjustable AI settings.
In this guide you are going to learn how to install and setup Kobold AI and use it in your web browser. You will also learn how to use the URL in Janitor AI or in Venus AI for chatting with the characters.
This tutorial does not need any technical knowledge required to accomplish. You can setup this very easily and in minutes you will have your own Kobold AI API up and running.
Setup Kobold AI in Colab for Free
Colab is a research tool built by Google over Jupyter notebook which can be used to deploy Python or R language based models or scripts. You will also get a Nvidia Tesla T4 GPU for free. This GPU costs around $250 in Google Cloud.
Now we shall setup Kobold AI in Google Colab.
Go to the the link with Kobold AI with GPU.
Play Audio File to Keep Tab Alive
You will see a welcome text “Welcome to KoboldAI on Google Colab, GPU Edition!“
Scroll to the section as shown below.
If you are using mobile phone to do this operation, you need click the play button to generate an audio. This is because the deployment takes some time and also to keep the tab alive in your browser, you need to play the audio.
Click the play icon and then start playing the audio.
You will get a warning about “This notebook was not authored by Google“
Click Run anyway.
Waif a little bit so that you can see a tick symbol near the pay button which indicates the execution is completed.
Now you will see an audio player. Play the audio.
Install Kobold AI API – Official
Now in the next section you need to choose the desired model that suits you. Below in the table are given some descriptions about the models.
Choose one of the listed model you need.
Update Code to include other Models
Click the show code button and include the models Nerybus 6B and Erebus 6B
Model = “Nerys V2 6B” #@param [“Nerys V2 6B”, “Skein 6B”, “Janeway 6B”, “Adventure 6B”, “Nerys 2.7B”, “AID 2.7B”, “Janeway 2.7B”, “Picard 2.7B”, “OPT 2.7B”, “Fairseq Dense 2.7B”, “Neo 2.7B”, “Nerybus 6B”, “Erebus 6B”] {allow-input: true}
Update the if condition below with the extra conditions and you are ready to use other models.
if Model == “Nerys V2 6B”: Model = “KoboldAI/OPT-6B-nerys-v2” path = “” download = “” #extra codes to be added elif Model == “Nerybus 6B”: Model = “KoboldAI/OPT-6.7B-Nerybus-Mix” path = “” download = “” elif Model == “Erebus 6B”: Model = “KoboldAI/OPT-6.7B-Erebus” path = “” download = “”
If you don’t need to use Google Drive, you can check off that (optional). If you want to use Google Drive, you need to allow access when prompted.
Click on the Play button to start the deployment.
The deployment will get started with the below output of the GPU type that is used for Kobold AI
+-+ | NVIDIA-SMI 525.85.12 Driver Version: 525.85.12 CUDA Version: 12.0 | |-+-+-+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |==============================+=====================+======================| | 0 Tesla T4 Off | 00000000:00:04.0 Off| 0 | | N/A 37C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | | | | N/A | +-+-+-+ +-+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |===========================================================================| | No running processes found | +-+
Wait for some time for the deployment to complete by setting up all drivers and packages that are required to run Kobold AI API.
Once the deployment is completed you will get an output similar to the one below, from there you will get a URL to check your Kobold AI in your browser.
INIT | Starting | LUA bridge INIT | OK | LUA bridge INIT | Starting | LUA Scripts INIT | OK | LUA Scripts INIT | OK | Webserver MESSAGE | KoboldAI has finished loading and is available at the following link: https://petersburg-source-shark-blade.trycloudflare.com
In the MESSAGE you will see your URL. This output the generated when you have the Version chosen as Official.
You will have the old UI that looks like one below.
Herr you will see all the details of Kobold AI like Connection status, Model that is used and many more.
Install Kobold AI United
To use the new UI in Kobold UI United, you just need to make a single change in your settings before the deployment.
Choose Version as United.
Click the Play button.
Once the deployment is completed you will get the following URLs
- Kobold AI Old UI
- Kobold AI New UI
- Kobold AI Lite
- Kobold AI API URL
Once you have received the URLs you need to wait for sometime for the tensors to be loaded. Once done you can start using the new UI.
INFO | modeling.inference_models.hf:set_input_parameters:191 – {‘use_gpu’: True, ‘0_Layers’: 32, ‘CPU_Layers’: 0, ‘Disk_Layers’: 0, ‘id’: ‘KoboldAI/OPT-6B-nerys-v2’, ‘model’: ‘KoboldAI/OPT-6B-nerys-v2’, ‘path’: None, ‘menu_path’: ”} INIT | Starting | Flask INIT | OK | Flask INIT | Starting | Webserver INIT | OK | Webserver MESSAGE | KoboldAI has finished loading and is available at the following link for UI 1: https://claimed-g-jokes-undertaken.trycloudflare.com MESSAGE | KoboldAI has finished loading and is available at the following link for UI 2: https://claimed-g-jokes-undertaken.trycloudflare.com/new_ui MESSAGE | KoboldAI has finished loading and is available at the following link for KoboldAI Lite: https://claimed-g-jokes-undertaken.trycloudflare.com/lite MESSAGE | KoboldAI has finished loading and is available at the following link for the API: https://claimed-g-jokes-undertaken.trycloudflare.com/api
Your Kobold AI new UI looks similar to the one below.
That’s all, now you have your Kobold AI server up and running.
Clean up Colab Workspace
Once you have completed your experiments you need to cleanup Colab to save resources. Please follow the below steps to perform a cleanup.
- Click the stop button (earlier it was a play button) to stop the web server.
- Go to Runtime and click Disconnect and delete runtime.
Now you are good to go. If you wish to setup up again, you can try the same setup as mentioned above.
Conclusion
Now you have learned how to setup Kobold AI in Google Colab for free with Nvidia T4 Tesla GPU.
How to Install Kobold AI API: Easy Step-by-Step Guide