Craft Your Own AI-Powered Knowledge Base: Wordpress + Ollama + AnythingLLM

Need your knowledge base to unlock its true potential?

This guide provides step-by-step instructions, from setting up WordPress and Ollama, to configuring AnythingLLM and interacting with your data.

By the end of this tutorial, you’ll have a robust system capable of integrating your proprietary knowledge base with an open-source language model.

Part 1: Wordpress Setup

In this step, we’ll create a dedicated virtual machine (VM) on Google Cloud Platform (GCP) to run Wordpress, the tool that acts as your company’s knowledge base.

Step 1: Deploy a WordPress Virtual Machine (VM) on GCP

1. Head to the GCP Console and create a new instance. Name it wordpress and choose your preferred region and zone. For consistency, we’ll be using us-central1 and us-central-a in this project.
2. Set the Boot Disk to Ubuntu 20.04 LTS.
3. Under Firewall, allow both HTTP and HTTPS traffic.
4. Go to Advanced Options > Networking
5. Toggle the default network interface.
6. Under External IPv4 address, click “Reserve Static External IP Address”.

Name it wordpress-ip and click “Reserve”. You should now see an external IP displayed.

Click “Done” and then “Create” to launch your VM.

Why the static IP?

We’ll need a constant alias:IP combination when setting up AnythingLLM later.

Step 2: Install Docker and WordPress

1. SSH into your newly created VM instance.
2. Install Docker: Follow the official guide for Ubuntu: https://docs.docker.com/engine/install/ubuntu/
3. Install WordPress using Docker Compose:

• Clone the repository:

git clone https://github.com/docker/awesome-compose.git

• Navigate to the WordPress folder:

cd awesome-compose/wordpress-mysql

• Deploy Docker Compose:

sudo docker compose up -d

Note: The compose.yaml file uses mariadb:10.6.4-focal. While stable, consider updating to the latest MariaDB version (10.11 or higher) by modifying the version in the compose.yaml file before deployment.

And that’s it! You now have a fully operational WordPress and MariaDB database.

To finalize your WordPress setup, visit the external IP of your VM and follow the instructions provided by the interface.

Part 2: Ollama Setup

In this step, we’ll create a dedicated virtual machine (VM) on Google Cloud Platform (GCP) to run Ollama, the tool that hosts and runs your LLM of choice.

Step 1: Launch Your Ollama VM on GCP

1. Head to the GCP Console and create a new instance. Name it ollama and choose your preferred region and zone. For consistency, we’ll be using us-central1 and us-central-a in this project.
2. Machine Configuration:

• Performance: Choose a machine type based on your LLM model’s needs.
• Recommendation: For a decent experience, use a minimum of “n2-standard-8” (8 vCPUs, 2 cores, 32 GB memory).
• Cost Optimization: Consider “spot” provisioning for potential cost savings during testing.

3. Set the Boot Disk to Ubuntu 20.04 LTS.

4. Allocate at least 20 GB for the boot disk size, accommodating Ollama’s and llama2:chat’s download size (7 GB).

5. Under Firewall, allow both HTTP and HTTPS traffic.

6. Click “Create” to launch your VM.

Step 2: Ollama & Llama2:chat Installation

1. SSH into the instance you just created.
2. Install Ollama by running the following command:

curl https://ollama.ai/install.sh | sh

3. Verify that Ollama is running:

sudo systemctl status ollama

4. Download and run llama2:chat:

ollama run llama2:chat

5. To exit running Llama2, use ctrl + d.

Please note that when using Ollama with AnythingLLM, only the chat models are available as per the GitHub documentation.

Step 3: Configure Ollama for AnythingLLM Communication (if necessary)

This step ensures that Ollama communicates effectively with AnythingLLM.

It’s important because sometimes (50/50) Ollama starts running on localhost (127.0.0.1), which AnythingLLM cannot access directly.

How to check if you need this step?

Upon installation, if you see the below, continue to the next step (Part 3):

“The Ollama API is now available at 0.0.0.0:11434.”

If you see this, “The Ollama API is now available at 127.0.0.1:11434”, proceed with the configuration below.

To resolve this issue, you need to modify the ollama.service file to allow Ollama to listen on all interfaces (0.0.0.0). This can be achieved by adding an environment variable to the [Service] section of the ollama.service file.

Here are the steps to do this:

1. Stop the Ollama service:

sudo systemctl stop ollama

2. Edit the service configuration:

sudo systemctl edit ollama.service

This command opens an editor for the ollama.service.d/override.conf file.

3. Add environment variables: In the opened editor, add the following lines under the [Service] section:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Please note that you should not omit the quotes around the environment variable values.

4. After editing the configuration, reload the systemd daemon and restart the Ollama service:

sudo systemctl daemon-reload
sudo systemctl restart ollama.service

5. Verify the service status:

sudo systemctl status ollama.service

6. Check open ports:

sudo ss -tuln

You should see port 11434 open and ready for communications on any interface and to any remote IP. This configuration ensures that AnythingLLM can recognize and interact with Ollama as expected.

With these changes, Ollama is now ready to communicate with the AnythingLLM instance.

Part 3: AnythingLLM Setup

Step 1: Launch via Deployment Manager

Begin by navigating to the GCP deployment section of the GitHub repository.

The steps are outlined for you, but they require some modifications to work correctly (only tested on Ubuntu).

Optionally, visit my Github repository to copy and paste the complete and correct code.

Firstly, you need to edit the provided deployment yaml file.

Open your editor, create a new file named ‘gcp_deploy_anything_llm_with_env.yaml’, and paste the raw content into it.

The zone is set as ‘us-central1-a’. If this zone does not match the region you’ve been using, adjust it accordingly.

Additionally, you need to add a line of code.

I discovered an error that causes the Docker container to start but then exit due to an error.

Add the following line to your file:

mkdir -p /home/anythingllm
touch /home/anythingllm/.env
sudo chown -R ubuntu:ubuntu /home/anythingllm # add this

Next, open your console and the shell. Run the following commands:

1. Authenticate with GCP:

gcloud auth login

2. Check your current region:

gcloud config get-value compute/region

3. Set your region to us-central1:

gcloud config set compute/region us-central1

4. Upload the yaml file from your computer.

5. Run the command from your default folder:

gcloud deployment-manager deployments create anything-llm-deployment - config gcp_deploy_anything_llm_with_env.yaml

6. View the status of the deployment:

gcloud compute instances get-serial-port-output anything-llm-instance - zone us-central1-a

You’ll see a message at the bottom indicating where the next output will start.

For example, “Specify — start=201353 in the next get-serial-port-output invocation to get only the new output starting from here.”

Use this value to check the status:

gcloud compute instances get-serial-port-output anything-llm-instance - zone us-central1-a - start=201353

The process typically takes about 5 minutes to complete.

When finished, visit the external IP of your instance and append port 3001 to the end. For example, (external_ip):3001.

Step 2: Interface Configuration

AnythingLLM provides a straightforward onboarding process that guides you through the setup.

1. Select your LLM preference: Choose ‘Ollama’ from the list and input the IP and port of your Ollama instance in the format (external_ip):11434. For example, http://35.193.146.170:11434.

If Ollama was configured correctly in the previous step, ‘llama2:chat’ will appear in the box. Enter your desired token context window.

2. Choose your embedding preference: The AnythingLLM Embedder is the default option and requires no additional configuration.

3. Choose the vector database connection: LanceDB is the default option and requires no additional configuration. However, other options such as Chroma, Pinecone, and others are available.

4. Choose a custom logo or proceed to the next step: You have the option to personalize your interface with a custom logo.

5. Configure the user settings: AnythingLLM can be configured for individual use or team setup.

6. Create a password: Secure your setup by creating a strong password.

7. Review the data handling and privacy settings: Familiarize yourself with how AnythingLLM handles data and privacy.

8. Participate in the survey or skip to the next step: You have the option to provide feedback through a survey.

9. Create a workspace: A workspace allows you to customize various configuration settings such as custom prompt instructions, LLM temperature, and more.

With these steps, you’re all set to chat with your LLM.

However, one more step is required to finalize the connection with your WordPress knowledge base.

Step 3: Adding a Wordpress Alias

Earlier in the tutorial, we established a static IP address for WordPress. Now, we’ll put it to use.

In WordPress, I published a blog that includes text from two pages of a Disney Investor Relations report.

Since we haven’t configured a domain name, the blog is currently accessible only via the WordPress instance IP.

However, when we add this IP to the AnythingLLM interface, it encounters issues with reading and parsing the IP.

To resolve this, we’ll create an alias for WordPress in a new AnythingLLM Docker container. Here’s how:

1. SSH into your AnythingLLM instance.
2. Retrieve the current container ID:

sudo docker ps

3. Stop the container:

sudo docker stop (container_id)

4. Remove the container to maintain a clean instance and free up space:

sudo docker rm (container_id)

5. In the GCP console, locate the external IP of your WordPress instance. Then, add it to the following command (the same command from your deployment yaml):

--add-host=wordpress:(external_ip)

6. Run the following command:

sudo docker run -d -p 3001:3001 - cap-add SYS_ADMIN -v /home/anythingllm:/app/server/storage -v /home/anythingllm/.env:/app/server/.env -e STORAGE_DIR="/app/server/storage" - add-host=wordpress:34.136.25.92 mintplexlabs/anythingllm:master

7. Verify that the new container is up and running and view the new container ID:

sudo docker ps

8. Use the exec command on the Docker container to confirm your alias was added:

sudo docker exec -it (container_id) cat /etc/hosts

While you could avoid all these steps by adding the WordPress IP into the deployment yaml from the start, it’s beneficial to understand what could go wrong and how to troubleshoot it.

Step 4: Adding a WordPress Blog & Confirming in Vector Database

Now that our setup is operational, let’s feed the WordPress blog data to AnythingLLM and verify its readability.

1. Navigate to your AnythingLLM web interface by entering (external_ip):3001 in your browser.
2. Access your workspace.

1. Click on ‘Settings’.
2. Copy your WordPress URL and replace it with the WordPress alias. For example, change http://34.136.25.92/disney-executive-officers/ to http://wordpress/disney-executive-officers/.

5. Click on ‘Fetch website’.

6. Move the fetched file to the workspace.

7. Click on ‘Save and Embed’.

8. SSH into your AnythingLLM instance.

9. Navigate to the custom documents folder:

cd /home/anythingllm/documents/custom-documents

10. View the contents of the file:

cat (file_name)

We’ve confirmed that AnythingLLM can read our WordPress alias, and that the knowledge base article has been successfully embedded.

You’re now ready to talk with your data!

Step 5: Interacting with Your Data

Now comes the exciting part: engaging with your AI-powered knowledge base!

Understanding LLM Interaction

• Google Gemini API: This model might prioritize its existing knowledge over the specific document you added. Be more specific in your prompts to guide it towards the document content.
• Llama2:chat: This model requires very precise prompts and questions to deliver accurate responses based on your document.

Remember, the responses you receive will depend on the LLM model you chose.

Exploring Interaction Modes in AnythingLLM

AnythingLLM offers two modes for interacting with your LLM:

1. Chat Mode: Provides a conversational, free-flowing interaction similar to GPT-based models. Ask open-ended questions or initiate a dialogue naturally.
2. Query Mode: Demands more specific, direct questions to receive focused answers based on the information in your document.

Testing Document Recognition

Let’s put your setup to the test! Find a unique piece of information within your WordPress blog and try these approaches:

Chat Mode:

Snippet from our WordPress blog

Query Mode:

Snippet from our WordPress blog

Success!

AnythingLLM correctly identifies and responds to your questions using information from your WordPress document, and you’ve successfully established your AI-powered knowledge base.

Bonus Feature: Notice the “citations” feature?

This displays which documents the LLM used to answer your queries, aiding in understanding and verification. This is particularly useful when you have multiple documents cached in your workspace.

Conclusion and Future Directions

Congratulations! You’ve successfully established a self-hosted knowledge base, set up Ollama to run llama2:chat, and configured AnythingLLM to read and vectorize your knowledge base content.

While this tutorial laid the foundation, remember that security is paramount. Before using your company’s knowledge base in production, consider the following next steps:

1. Prioritize Privacy:

• Build a Secure Network: Construct a private Virtual Private Cloud (VPC) environment with isolated subnets and robust firewall rules. This restricts access and safeguards your data against unauthorized intrusion and exfiltration.
• Data Protection Measures: Implement strict IAM permissions and access control mechanisms to further bolster the security of your knowledge base content.

2. Explore Advanced Customization: ️

• Fine-tune AnythingLLM Configuration: Dive deeper into AnythingLLM’s settings to personalize its responses and tailor them to your specific needs. Explore different LLM models and fine-tune parameters to achieve optimal results.
• Expand Your Data Sources: Consider integrating additional data sources beyond your WordPress blog (such as Github) to enrich your knowledge base and diversify its capabilities.

3. Continuous Learning and Optimization:

• Experiment and Refine: Regularly test your LLM with diverse queries and assess its performance. Adjust prompts, explore different interaction modes, and gather feedback to refine its accuracy and usability.
• Stay Updated: Keep your system up-to-date with the latest software versions and security patches to ensure optimal performance and protection.

By following these steps, you can transform your knowledge base into a truly secure and powerful tool, empowering you with information and insights readily at your team’s fingertips.