Your Web News in One Place

Help Webnuz

Referal links:

Sign up for GreenGeeks web hosting
November 17, 2022 09:00 am GMT

Build a Chatbot Using Python, Django

A ChatBot has become one of the must-have features of modern-day web applications. It plays a huge role in customer service which was once traditionally held by human beings. Through ChatBot, you can automatically generate a response to a users input by making use of different machine-learning techniques.

Note:- If you encounter any issues throughout the tutorial, you can check out the code in theGitHub repository.

Table of Contents

  1. Prerequisite
  2. Project Configuration
  3. WebSockets Using Django Channels
  4. Tailwind CSS Configuration
  5. Frontend - Create a WebSocket Connection
  6. Enable a Channel Layer
  7. Backend - WebSocket Consumers
  8. ChatterBot - Generate Automated Response
  9. ChatterBot Training
  10. Offload Automated Response Generation to Celery
  11. Conclusion

Prerequisite

This guide assumes that you have intermediate-level Django knowledge.

Project Configuration

To implement the ChatBot, the following technologies will be used:

  • ChatterBot Library - To generate automated responses to a users input.
  • Celery - Getting the automated response from a machine-learning dialog engine is going to take a while. So Celery will be used to perform this task in the background.
  • WebSockets using Django Channels - To send to the client the automated response generated by the machine learning model immediately when its available.
  • Redis - Will be used as a message broker and result backend for Celery. In addition, it will be used as a channel layer for WebSocket communication.
  • TailwindCSS - To create the user interface.

We will dive into the details of each of them, but first, lets set up the project. I have created a starter Docker template with all the configurations so that we can move forward with the development without wasting time.

Create a folder on your local machine > cd into it > clone the template:

mkdir chatbotcd chatbotgit clone https://github.com/earthcomfy/blog-template .

This will generate the following folders and files inside your project:

Template

There isnt much going on except a basic Django setup in Docker with web, db, redis, and celery services. Refer to this article for more information on how to set up a Django project inside Docker. If you dont want to use Docker, you can also follow along by setting up a virtual environment and running the necessary services on your machine.

In the root directory of your project, there is a file named .env.template, rename it to .env and update the environment variables accordingly.

Great, now build the image and spin up the containers by running:

docker-compose up --build

Go to http://0.0.0.0:8000/ to check if everything worked correctly.

Welcome

Great, everything is working. Now close Docker by pressing ctrl + c and create a Django app named chat

docker-compose run --rm web python manage.py startapp chat

Add it to the list of installed apps inside settings/base.py

# settings/base.pyINSTALLED_APPS = [    "django.contrib.admin",    "django.contrib.auth",    "django.contrib.contenttypes",    "django.contrib.sessions",    "django.contrib.messages",    "django.contrib.staticfiles",    # Local apps    "chat"]

Hurray! Lets move on to the next section where the fun begins.

WebSockets Using Django Channels

It goes without saying that in a typical Django project the client makes an HTTP request > Django calls the view that is responsible for managing this request and returns a response back to the client.

This communication is pretty standard. However, ever since Django introduced ASGI and started supporting it natively, writing Django applications with asynchronous code has become possible.ASGI not only allows you to run asynchronous apps, but it also makes it possible to work with more advanced protocols like WebSockets.

Django Channels, which is built on ASGI, goes beyond HTTP and support other protocols such as WebSockets. The underlying implementation of Django Channels is very similar to regular HTTP views.

That being said, the following packages are needed to set up channels in a project:

  • channels
  • daphne

Both of them were added to the requirements.txt file so they must have already been installed when you built your Docker image.

Daphneis an HTTP, HTTP2, and WebSocket protocol server for ASGI, and was developed to power Django Channels. This server needs to be added to the list of installed apps so that it can take over the runserver management command.

# settings/base.pyINSTALLED_APPS = [    "daphne", # here    "django.contrib.admin",    "django.contrib.auth",    "django.contrib.contenttypes",    "django.contrib.sessions",    "django.contrib.messages",    "django.contrib.staticfiles",    # Local apps    "chat"]

Note:- Please be wary of any other third-party apps that require an overloaded or replacementrunserver command. Daphne provides a separaterunserver command and may conflict with it. In order to solve such issues, make suredaphne is at the top of yourINSTALLED_APPS

Next, go to your project'sasgi.py file and adjust it as follows to wrap the Django ASGI application:

# asgi.pyimport osfrom channels.auth import AuthMiddlewareStackfrom channels.routing import ProtocolTypeRouter, URLRouterfrom channels.security.websocket import AllowedHostsOriginValidatorfrom decouple import configfrom django.core.asgi import get_asgi_applicationos.environ.setdefault("DJANGO_SETTINGS_MODULE", config("DJANGO_SETTINGS_MODULE"))# Initialize Django ASGI application early to ensure the AppRegistry# is populated before importing code that may import ORM models.django_asgi_app = get_asgi_application()application = ProtocolTypeRouter(    {        "http": django_asgi_app,        # We will add WebSocket protocol later. For now, it's just HTTP.    })

Finally, set yourASGI_APPLICATION setting to point to that routing object as your root application:

# settings/base.pyASGI_APPLICATION = "config.asgi.application"

Perfect! From this point onward, the daphne development server will be run. To ensure this, fire up the containers by running:

docker-compose up

ASGI

Notice the ASGI/Daphne version 4.0.0 development server

Alright, this application is going to require a single view where the user and the ChatBot can interact. For this, lets first set up Tailwind CSS in the project.

Tailwind CSS Configuration

There are many ways to set up Tailwind CSS in Django. One way to do so is by using a package called django-tailwind. This package provides an easy way to use Tailwind CSS in a Django project.

This package was included in requirements.txt so it must have already been installed when you created the Docker image.

After installation, add tailwind to the list of installed apps in the settings:

# settings/base.pyINSTALLED_APPS = [    "daphne",    "django.contrib.admin",    "django.contrib.auth",    "django.contrib.contenttypes",    "django.contrib.sessions",    "django.contrib.messages",    "django.contrib.staticfiles",    # Third-party apps    "tailwind",    # Local apps    "chat",]

Now, Create aTailwindCSS-compatible Django app. When prompted for the app name you can give it any name or proceed with the default theme

docker-compose run --rm web python manage.py tailwind init

Add the app - theme to the list of installed apps in the settings:

# settings/base.pyINSTALLED_APPS = [    # ...    # Local apps    "chat",    "theme",]

Register the generatedapp theme by adding it to the settings as follows:

# settings/base.py# TailwindTAILWIND_APP_NAME = "theme"

InstallTailwind CSSdependencies by running the following command:

docker-compose run --rm web python manage.py tailwind install

Perfect! Now, head over to theme/templates/base.html This file will be used as a base template to include Tailwinds style sheet in all the other templates. Lets modify it as follows:

{% comment %} theme/templates/base.html {% endcomment %}{% load static tailwind_tags %}<!DOCTYPE html><html lang="en">  <head>    <title>{% block title %}Django Chatbot{% endblock %}</title>    <meta charset="UTF-8" />    <meta name="viewport" content="width=device-width, initial-scale=1.0" />    <meta http-equiv="X-UA-Compatible" content="ie=edge" />    {% tailwind_css %}  </head>  <body>    {% block body %} {% endblock %}  </body>  {% block scripts%}{% endblock %}</html>

Then, head over to the chat app created earlier and create a templates directory. Within the templates directory, create another directory namedchat, and within that create a file named chat.html

Put the following snippet inside the file:

{% comment %} chat/templates/chat/chat.html {% endcomment %}{% extends 'base.html' %} {% block body %}<div class="p-6 w-[800px]">  <h1 class="text-3xl tracking-tight font-light" id="chat-header"></h1>  <div    id="chat-log"    class="mt-4 w-full relative p-6 overflow-y-auto h-[30rem] bg-gray-50 border border-gray-200"  ></div>  <div class="mt-4">    <input      id="chat-message-input"      class="py-2 outline-none bg-gray-50 border border-gray-300 text-gray-900 text-sm focus:border-blue-500"      type="text"      placeholder="Write your message here."    />    <button      id="chat-message-submit"      class="py-2 px-4 ml-2 border border-transparent text-sm font-medium rounded-md text-white bg-blue-800 hover:bg-blue-900"      type="submit"    >      Send    </button>  </div></div>{% endblock %}

The above is a basic Django HTML file with a div where chat history will be displayed and an input box for the users input with the submit button.

Next, create a view that will render the above template:

# chat/views.pyfrom django.views.generic import TemplateViewclass ChatView(TemplateView):    template_name: str = "chat/chat.html"

Then, Createurls.py module inside the chat app and map theChatView to the URL patterns.

# chat/urls.pyfrom django.urls import pathfrom .views import ChatViewapp_name = "chat"urlpatterns = [path("", ChatView.as_view(), name="chat_view")]

Finally, go to the project'surls.pyfile and include the chat app's URL:

# config/urls.pyfrom django.contrib import adminfrom django.urls import include, pathurlpatterns = [    path("admin/", admin.site.urls),    path("", include("chat.urls", namespace="chat")),]

Done! Now, make sure the styles are applied properly and that what we have so far is working.

Start tailwind:

docker-compose run web python manage.py tailwind start

In another terminal spin up the containers:

docker-compose run web python manage.py tailwind start

Go to http://localhost:8000/ and you will see the following

chat view

Note:- If the styles arent applied, make sure you have started tailwind and restart the containers.

Frontend - Create a WebSocket Connection

Inside chat.html, you need to write a script that will open a WebSocket connection to a Django Server and listen to the established connection to send and receive data.

Open the chat.html file and add the following script just under the body block i.e. {% endblock %}

{% block scripts %}<script>  var wss_protocol = window.location.protocol == "https:" ? "wss://" : "ws://";  var chatSocket = new WebSocket(    wss_protocol + window.location.host + "/ws/chat/"  );  var messages = [];  chatSocket.onopen = function (e) {    document.querySelector("#chat-header").innerHTML =      "Welcome to Django Chatbot";  };  chatSocket.onmessage = function (e) {    var data = JSON.parse(e.data);    var message = data["text"];    messages.push(message);    var str = '<ul class="space-y-2">';    messages.forEach(function (msg) {      str += `<li class="flex ${        msg.source == "bot" ? "justify-start" : "justify-end"      }">      <div class="relative max-w-xl px-4 py-2 rounded-lg shadow-md        ${          msg.source == "bot"            ? "text-gray-700 bg-white border border-gray-200"            : "bg-blue-600 text-white"        }">        <span className="block font-normal">${msg.msg}</span></div></li>`;    });    str += "</ul>";    document.querySelector("#chat-log").innerHTML = str;  };  chatSocket.onclose = function (e) {    alert("Socket closed unexpectedly, please reload the page.");  };  document.querySelector("#chat-message-input").focus();  document.querySelector("#chat-message-input").onkeyup = function (e) {    if (e.keyCode === 13) {      // enter, return      document.querySelector("#chat-message-submit").click();    }  };  document.querySelector("#chat-message-submit").onclick = function (e) {    var messageInputDom = document.querySelector("#chat-message-input");    var message = messageInputDom.value;    chatSocket.send(      JSON.stringify({        text: message,      })    );    messageInputDom.value = "";  };</script>{% endblock %}
  • The above script creates a WebSocket connection to the /ws/chat/ path > listens to different events and manipulates the DOM.
  • Depending on the source of the message (user or bot), different alignments and styles are applied (see under the chatsocket.onmessage)

If you run the server at this point, type in any message, and hit submit, the WebSocket connection will be opened but you will see an error in the JavaScript console because we dont have a consumer that accepts WebSocket connections yet.

Enable a Channel Layer

A Channel layer provides an abstraction for multiple consumers to talk with each other and with other parts of Django. It is the middleman that passes messages from senders to receivers. The Channel layer that will be used in this guide is Redis. Aside from serving as a WebSocket communication, Redis can also be used as a:

  • Message Broker and Result backend for Celery.
  • Caching.

It is an all-in-one tool!

We have already configured a Redis service inside Docker. Django Channels also needs to know how to interface with Redis. This is provided by a package called channels_redis

channels_redis is an officially maintained channel layer that uses Redis as its backing store. It is already included in requirements.txt so it must have already been installed when you built your Docker image.

Next, add the Channel layer settings:

# settings/base.py# Django ChannelsCHANNEL_LAYERS = {    "default": {        "BACKEND": "channels_redis.core.RedisChannelLayer",        "CONFIG": {            "hosts": [config("REDIS_BACKEND")],        },    },}

A channel layer has the following:

1) Channel- a mailbox where messages can be sent to. Each channel has a name. Anyone who has the name of a channel can send a message to the channel.

2)Group- a group of related channels. A group has a name. Anyone who has the name of a group can add/remove a channel to the group by name and send a message to all channels in the group.

Backend - WebSocket Consumers

When a WebSocket API opens a connection, Channels need to be informed where to look to accept and handle the connection. This is where Consumers come in. Consumers correspond to Views in normal Django HTTP requests.

When a message is sent from the client, it is received by the consumers listening to the group or channel on the other end.

Create a file named consumers.py inside the chat app and add the following:

# chat/consumers.pyimport jsonfrom asgiref.sync import async_to_syncfrom channels.generic.websocket import WebsocketConsumerclass ChatConsumer(WebsocketConsumer):    def receive(self, text_data):        text_data_json = json.loads(text_data)        # The consumer ChatConsumer is synchronous while the channel layer        # methods are asynchronous. Therefore wrap the methods in async-to-sync        async_to_sync(self.channel_layer.send)(            self.channel_name,            {                "type": "chat_message",                "text": {"msg": text_data_json["text"], "source": "user"},            },        )        # We will later replace this call with a celery task that will        # use a Python library called ChatterBot to generate an automated        # response to a user's input.        async_to_sync(self.channel_layer.send)(            self.channel_name,            {                "type": "chat.message",                "text": {"msg": "Bot says hello", "source": "bot"},            },        )    # Handles the chat.mesage event i.e. receives messages from the channel layer    # and sends it back to the client.    def chat_message(self, event):        text = event["text"]        self.send(text_data=json.dumps({"text": text}))
  • The above consumer accepts a WebSocket connection on the path /ws/chat/, takes a message, and sends a response to the client (for now the response is just a simple message Bot says hello).
  • Any consumer has aself.channel_layerandself.channel_nameattribute, which contains a pointer to the channel layer instance (Redis in our case) and the channel name respectively.

Similar to urls.py Channels uses a routing configuration to route to the consumer. Inside the chat app, create a file named routing.py and add the following:

# chat/routing.pyfrom django.urls import re_pathfrom . import consumerswebsocket_urlpatterns = [    re_path(r"ws/chat/$", consumers.ChatConsumer.as_asgi()),]

Next, head over toasgi.pyand modify it as follows:

# config/asgi.py"""ASGI config for config project.It exposes the ASGI callable as a module-level variable named ``application``.For more information on this file, seehttps://docs.djangoproject.com/en/4.1/howto/deployment/asgi/"""import osfrom channels.auth import AuthMiddlewareStackfrom channels.routing import ProtocolTypeRouter, URLRouterfrom channels.security.websocket import AllowedHostsOriginValidatorfrom django.core.asgi import get_asgi_applicationfrom decouple import configos.environ.setdefault("DJANGO_SETTINGS_MODULE", config("DJANGO_SETTINGS_MODULE"))# Initialize Django ASGI application early to ensure the AppRegistry# is populated before importing code that may import ORM models.django_asgi_app = get_asgi_application()import chat.routingapplication = ProtocolTypeRouter(    {        "http": django_asgi_app,        "websocket": AllowedHostsOriginValidator(            AuthMiddlewareStack(URLRouter(chat.routing.websocket_urlpatterns))        ),    })
  • The above code points the root routing configuration to thechat.routingmodule. This means when a connection is made to the development server (channel's development server),ProtocolTypeRouterchecks whether it's a normal HTTP request or a WebSocket.
  • If it's a WebSocket,AuthMiddlewareStackwill take it from there and will populate the connections scope with a reference to the currently authenticated user, similar to how DjangosAuthenticationMiddlewarepopulates therequestobject of aviewfunction with the currently authenticated user.
  • Next,URLRouterwill route the connection to a particular consumer based on the provided URL patterns.

Now, if you head over to http://localhost:8000/ and type in any message, you will get a response that says Bot says hello

chat response

Great! Now, the response to a users input needs to be automated. Lets do that in the next section using ChatterBot.

ChatterBot - Generate Automated Response

ChatterBot is a Python library that makes it easy to generate automated responses to a users input. To use this in your project, add it to the list of installed apps in the settings:

# settings/base.pyINSTALLED_APPS = [    # ...    # Third-party apps    "tailwind",    "chatterbot.ext.django_chatterbot",    # Local apps    "chat",    "theme",]

Under the hood, ChatterBot uses logic adapters to determine how to respond to a given input statement. You can have multiple logic adapters for your bot. Most of the logic adapters use the Naive Bayesian classification algorithm to determine if an input statement meets a particular set of criteria and to generate a response accordingly. The Naive Bayesian classification algorithm is based on conditional probability. Learn more about it here.

For a list of logic adapters, you can check this out. For the purpose of this tutorial, lets use a single adapter called Best Match Adapter. As the name implies, theBestMatch logic adapter selects a response based on the best-known match to a given statement. Lets add it to the settings:

# settings/base.py# ChatterbotCHATTERBOT = {    "name": "User Support Bot",    "logic_adapters": [        "chatterbot.logic.BestMatch",    ],}

If you use multiple adapters, the response with the highest calculated confidence score will be considered a valid response to the input. The confidence score returned by each logic adapter determines how valid or close the response is to the output.

The following diagram from ChatterBot documentation provides a very nice flow of what goes behind the scenes.

logic adapters

ChatterBot Training

In the beginning, a ChatterBot instance is like a blank slate. This means that it starts off with an empty database. You fill it with statements (a single string of text representing something that can be said) and gradually it learns to reply to your input with higher accuracy. This training process is going to take some time. In order to simplify this process, ChatterBot provides multiple ways to initially train your bot. In this tutorial, you will learn how to train your bot using sample list data.

Lets create a simple custom management command where we will pass in a list of statements. Head over to the chat app and create a management directory. Within the management directory, create another directory named commands, and within that create 2 files __init__.py and train.py

Put the following script inside train.py

# chat/management/commands/train.pyfrom django.core.management.base import BaseCommandfrom chatterbot import ChatBotfrom chatterbot.ext.django_chatterbot import settingsfrom chatterbot.trainers import ListTrainerclass Command(BaseCommand):    help = "Training the chatbot"    def handle(self, *args, **options):        chatterbot = ChatBot(**settings.CHATTERBOT)        trainer = ListTrainer(chatterbot)        trainer.train(            [                "Hello",                "Hi there!",                "How are you doing?",                "I'm doing great.",                "That is good to hear",                "Thank you.",                "You're welcome.",            ]        )        self.stdout.write(self.style.SUCCESS("Successfull!"))

Next, run the following command to load the example dialog into your projects database.

docker-compose run --rm web python manage.py train

Now, create a superuser:

docker-compose exec web python manage.py createsuperuser

and go to the admin panel http://localhost:8000/admin/ Go to the statements tab and voila, you will see a list of texts with their responses.

Statements

Offload Automated Response Generation to Celery

Needless to say, generating a response from ChatterBot is time-intensive. This is where Celery comes to the rescue. Celery will pick up queued tasks and manages a separate server to run them in the background. As mentioned before, we will use Redis as a message broker. It is already set up in Docker so lets proceed to defining the task.

Create a file named tasks.py inside the chat app and put the following Celery task:

# chat/tasks.pyfrom asgiref.sync import async_to_syncfrom celery import shared_taskfrom channels.layers import get_channel_layerfrom chatterbot import ChatBotfrom chatterbot.ext.django_chatterbot import settingschannel_layer = get_channel_layer()@shared_taskdef get_response(channel_name, input_data):    chatterbot = ChatBot(**settings.CHATTERBOT)    response = chatterbot.get_response(input_data)    response_data = response.serialize()    async_to_sync(channel_layer.send)(        channel_name,        {            "type": "chat.message",            "text": {"msg": response_data["text"], "source": "bot"},        },    )

Finally, update consumers.py with the following content:

# chat/consumers.pyimport jsonfrom asgiref.sync import async_to_syncfrom channels.generic.websocket import WebsocketConsumerfrom .tasks import get_responseclass ChatConsumer(WebsocketConsumer):    def receive(self, text_data):        text_data_json = json.loads(text_data)        get_response.delay(self.channel_name, text_data_json)        async_to_sync(self.channel_layer.send)(            self.channel_name,            {                "type": "chat_message",                "text": {"msg": text_data_json["text"], "source": "user"},            },        )    def chat_message(self, event):        text = event["text"]        self.send(text_data=json.dumps({"text": text}))

You have to restart the container so that Celery can pick up the task and add it to its queue. Press ctrl + c and run:

docker-compose up

Go to http://localhost:8000/ and type in hello and you will get the response back

message from chatterbot

Conclusion

This tutorial is intended to be a starting place. The ChatterBot documentation is very concise so make sure to check it out to implement more features. For instance, you can train your data with corpus data. You can use multiple logic adapters or even create your own. In addition, check out Django Channels documentation to understand more about consumers, channel layers, and so on.

If you got lost somewhere throughout the guide, check out the project onGitHub

Happy coding!


Original Link: https://dev.to/documatic/build-a-chatbot-using-python-django-46hb

Share this article:    Share on Facebook
View Full Article

Dev To

An online community for sharing and discovering great ideas, having debates, and making friends

More About this Source Visit Dev To