How to Set Up a Full-Text Search Using Scout in Laravel

Full-text search is crucial for allowing users to navigate content-rich websites. In this post, I'll show you how to implement full-text search for a Laravel app. In fact, we'll use the Laravel Scout library, which makes implementation of full-text search easy and fun.

What exactly is the Laravel Scout? The official documentation sums it up like this:

Laravel Scout provides a simple, driver-based solution for adding full-text search to your Eloquent models. Using model observers, Scout will automatically keep your search indexes in sync with your Eloquent records.

Basically, Laravel Scout is a library which manages manipulation of the index whenever there's a change in the model data. The place where the data will be indexed depends on the driver which you've configured with the Scout library.

As of now, the Laravel Scout library supports Algolia, a cloud-based search engine API, and that's what we'll use in this article to demonstrate the full-text search implementation.

We'll start by installing the Scout and Algolia server libraries, and as we move on we'll go through a real-world example to demonstrate how you could index and search your data.

Server Configurations

In this section, we're going to install the dependencies that are required in order to make the Scout library work with Laravel. After installation, we'll need to go through quite a bit of configuration so that Laravel can detect the Scout library.

Let's go ahead and install the Scout library using Composer.

$composer require laravel/scout

That's pretty much it as far as the Scout library installation is concerned. Now that we've installed the Scout library, let's make sure that Laravel knows about it.

Working with Laravel, you're probably aware of the concept of a service provider, which allows you to configure services in your application. Thus, whenever you want to enable a new service in your Laravel application, you just need to add an associated service provider entry in config/app.php.

If you're not familiar with Laravel service providers yet, I would strongly recommend that you do yourself a favor and go through this introductory article that explains the basics of service providers in Laravel.

In our case, we just need to add the ScoutServiceProvider provider to the list of service providers in config/app.php, as shown in the following snippet.

...
...
'providers' => [
    /*
     * Laravel Framework Service Providers...
     */
    Illuminate\Auth\AuthServiceProvider::class,
    Illuminate\Broadcasting\BroadcastServiceProvider::class,
    Illuminate\Bus\BusServiceProvider::class,
    Illuminate\Cache\CacheServiceProvider::class,
    Illuminate\Foundation\Providers\ConsoleSupportServiceProvider::class,
    Illuminate\Cookie\CookieServiceProvider::class,
    Illuminate\Database\DatabaseServiceProvider::class,
    Illuminate\Encryption\EncryptionServiceProvider::class,
    Illuminate\Filesystem\FilesystemServiceProvider::class,
    Illuminate\Foundation\Providers\FoundationServiceProvider::class,
    Illuminate\Hashing\HashServiceProvider::class,
    Illuminate\Mail\MailServiceProvider::class,
    Illuminate\Notifications\NotificationServiceProvider::class,
    Illuminate\Pagination\PaginationServiceProvider::class,
    Illuminate\Pipeline\PipelineServiceProvider::class,
    Illuminate\Queue\QueueServiceProvider::class,
    Illuminate\Redis\RedisServiceProvider::class,
    Illuminate\Auth\Passwords\PasswordResetServiceProvider::class,
    Illuminate\Session\SessionServiceProvider::class,
    Illuminate\Translation\TranslationServiceProvider::class,
    Illuminate\Validation\ValidationServiceProvider::class,
    Illuminate\View\ViewServiceProvider::class,

    /*
     * Package Service Providers...
     */

    /*
     * Application Service Providers...
     */
    App\Providers\AppServiceProvider::class,
    App\Providers\AuthServiceProvider::class,
    // App\Providers\BroadcastServiceProvider::class,
    App\Providers\EventServiceProvider::class,
    App\Providers\RouteServiceProvider::class,
    Laravel\Scout\ScoutServiceProvider::class,
],
...
...

Now, Laravel is aware of the ScoutServiceProvider service provider. The Scout library comes with a configuration file which allows us to set API credentials.

Let's go ahead and publish the assets provided by the Scout library using the following command.

$ php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"
Copied File [/vendor/laravel/scout/config/scout.php] To [/config/scout.php]
Publishing complete.

As you can see, it has copied the vendor/laravel/scout/config/scout.php file to config/scout.php.

Next, go ahead and create an account with Algolia as we'll need API credentials in the first place. Once you have the API information, let's go ahead and configure the necessary settings in the config/scout.php file, as shown in the following snippet.

<?php

return [

    /*
    |--------------------------------------------------------------------------
    | Default Search Engine
    |--------------------------------------------------------------------------
    |
    | This option controls the default search connection that gets used while
    | using Laravel Scout. This connection is used when syncing all models
    | to the search service. You should adjust this based on your needs.
    |
    | Supported: "algolia", "null"
    |
    */

    'driver' => env('SCOUT_DRIVER', 'algolia'),

    /*
    |--------------------------------------------------------------------------
    | Index Prefix
    |--------------------------------------------------------------------------
    |
    | Here you may specify a prefix that will be applied to all search index
    | names used by Scout. This prefix may be useful if you have multiple
    | "tenants" or applications sharing the same search infrastructure.
    |
    */

    'prefix' => env('SCOUT_PREFIX', ''),

    /*
    |--------------------------------------------------------------------------
    | Queue Data Syncing
    |--------------------------------------------------------------------------
    |
    | This option allows you to control if the operations that sync your data
    | with your search engines are queued. When this is set to "true" then
    | all automatic data syncing will get queued for better performance.
    |
    */

    'queue' => env('SCOUT_QUEUE', false),

    /*
    |--------------------------------------------------------------------------
    | Database Transactions
    |--------------------------------------------------------------------------
    |
    | This configuration option determines if your data will only be synced
    | with your search indexes after every open database transaction has
    | been committed, thus preventing any discarded data from syncing.
    |
    */

    'after_commit' => false,

    /*
    |--------------------------------------------------------------------------
    | Chunk Sizes
    |--------------------------------------------------------------------------
    |
    | These options allow you to control the maximum chunk size when you are
    | mass importing data into the search engine. This allows you to fine
    | tune each of these chunk sizes based on the power of the servers.
    |
    */

    'chunk' => [
        'searchable' => 500,
        'unsearchable' => 500,
    ],

    /*
    |--------------------------------------------------------------------------
    | Soft Deletes
    |--------------------------------------------------------------------------
    |
    | This option allows you to control whether to keep soft deleted records in
    | the search indexes. Maintaining soft deleted records can be useful
    | if your application still needs to search for the records later.
    |
    */

    'soft_delete' => false,

    /*
    |--------------------------------------------------------------------------
    | Identify User
    |--------------------------------------------------------------------------
    |
    | This option allows you to control whether to notify the search engine
    | of the user performing the search. This is sometimes useful if the
    | engine supports any analytics based on this application's users.
    |
    | Supported engines: "algolia"
    |
    */

    'identify' => env('SCOUT_IDENTIFY', false),

    /*
    |--------------------------------------------------------------------------
    | Algolia Configuration
    |--------------------------------------------------------------------------
    |
    | Here you may configure your Algolia settings. Algolia is a cloud hosted
    | search engine which works great with Scout out of the box. Just plug
    | in your application ID and admin API key to get started searching.
    |
    */

    'algolia' => [
        'id' => env('ALGOLIA_APP_ID', 'XXXXXX'),
        'secret' => env('ALGOLIA_SECRET', 'XXXXXXXXXXXXXXXXXXXXXXXX'),
    ],

];

Note that we've set the value of SCOUT_DRIVER to algolia driver. Thus, it's required that you configure the necessary settings for the Algolia driver at the end of the file. Basically, you just need to set the id and secret that you've got from the Algolia account.

As you can see, we're fetching values from environment variables. So let's make sure that we set the following variables in the .env file properly.

...
...
ALGOLIA_APP_ID=XXXXXX
ALGOLIA_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXX
...
...

Finally, we need to install the Algolia PHP SDK, which will be used to interact with Algolia using APIs. Let's install it using Composer, as shown in the following snippet.

$composer require algolia/algoliasearch-client-php

And with that, we've installed all the dependencies that are necessary in order to post and index data to the Algolia service.

How to Make Models Indexable and Searchable

In the previous section, we did all the hard work to set up the Scout and Algolia libraries so that we could index and search data using the Algolia search service.

In this section, we'll go through an example to demonstrate how you could index the existing data and retrieve search results from Algolia. I assume that you have a default Post model in your application.

The first thing that we'll need to do is to add the Laravel\Scout\Searchable trait to the Post model. That makes the Post model searchable; Laravel synchronizes post records with the Algolia index every time the post record is added, updated, or deleted.

<?php
namespace App;
 
use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;
 
class Post extends Model
{
    use Searchable;
    
    ...
    ...
}

With that, the Post model is search-friendly!

Next, we would like to configure the fields that should get indexed in the first place. Of course, you don't want to index all the fields of your model in Algolia to keep it effective and lightweight. In fact, more often than not, you won't need it.

You can add the toSearchableArray in the model class to configure the fields that'll be indexed.

/**
 * Get the indexable data array for the model.
 *
 * @return array
 */
public function toSearchableArray()
{
  $array = $this->toArray();
     
  return array('id' => $array['id'],'name' => $array['name']);
}

Now, we're ready to import and index existing Post records into Algolia. In fact, the Scout library makes this easy by providing the following artisan command.

$php artisan scout:import "App\Post"

That should import all the records of the Post model in a single go! They are indexed as soon as they're imported, so we're ready to query records already. Go ahead and explore the Algolia dashboard to see the imported records and other utilities.

How It Works Altogether

In this section, we'll create an example which demonstrates how to perform search and CRUD operations that are synced in real time with the Algolia index.

Go ahead and create the app/Http/Controllers/SearchController.php file with the following contents.

<?php
namespace App\Http\Controllers;
 
use App\Http\Controllers\Controller;
use App\Post;
 
class SearchController extends Controller
{
    public function query()
    {
        // queries to Algolia search index and returns matched records as Eloquent Models
        $posts = Post::search('title')->get();
         
        // do the usual stuff here
        foreach ($posts as $post) {
          // ...
        }
    }
 
    public function add()
    {
        // this post should be indexed at Algolia right away!
        $post = new Post;
        $post->setAttribute('name', 'Another Post');
        $post->setAttribute('user_id', '1');
        $post->save();
    }
     
    public function delete()
    {
        // this post should be removed from the index at Algolia right away!
        $post = Post::find(1);
        $post->delete();
    }
}

Of course, we need to add the associated routes as well.

Route::get('search/query', 'SearchController@query');
Route::get('search/add', 'SearchController@add');
Route::get('search/delete', 'SearchController@delete');

Let's go through the query method to see how to perform a search in Algolia.

public function query()
{
    // queries to Algolia search index and returns matched records as Eloquent Models
    $posts = Post::search('title')->get();
     
    // do the usual stuff here
    foreach ($posts as $post) {
        // ...
    }
}

Recall that we made the Post model searchable by adding the Searchable trait. Thus, the Post model can use the search method to retrieve records from the Algolia index. In the above example, we're trying to fetch records that match the title keyword.

Next, there's the add method, which imitates the workflow of adding a new post record.

public function add()
{
    // this post should be indexed at Algolia right away!
    $post = new Post;
    $post->setAttribute('name', 'Another Post');
    $post->setAttribute('user_id', '1');
    $post->save();
}

There's nothing fancy in the above code; it just creates a new post record using the Post model. But the Post model implements the Searchable trait, so Laravel does some extra work this time around by indexing the newly created record in Algolia. So as you can see, the indexing is done in real time.

Finally, there's the delete method. Let's go through it as well.

public function delete()
{
    // this post should be removed from the index at Algolia right away!
    $post = Post::find(1);
    $post->delete();
}

As you would have expected, the record is deleted right away from the Algolia index as soon as it's deleted from the database.

Basically, there's no extra effort required from your side if you want to make existing models searchable. Everything is handled by the Scout library using model observers.

How to Make a Custom Search Engine/Driver

By default, the Scout library supports the Algolia and MeiliSearch drivers. Additionally, you can also use the database driver for a lightweight database. On the other hand, if you want to implement your own custom engine, Scout allows you to achieve that. You just need to write your custom engine and register it with Scout!

Your custom engine class may look like this:

<?php

namespace App\Engines;

use Laravel\Scout\Builder;
use Laravel\Scout\Engines\Engine;

class CustomScoutEngine extends Engine {
    public function update($models) {}
    public function delete($models) {}
    public function search(Builder $builder) {}
    public function paginate(Builder $builder, $perPage, $page) {}
    public function mapIds($results) {}
    public function map(Builder $builder, $results, $model) {}
    public function getTotalCount($results) {}
    public function flush($model) {}
}

Of course, you need to implement abstract methods as per your requirements.

Once you've implemented your custom engine class, you just need to register it. You can do it with the help of the boot method of the service provider, as shown in the following snippet.

use App\Engines\CustomScoutEngine;
use Laravel\Scout\EngineManager;
 
/**
 * Bootstrap any application services.
 *
 * @return void
 */
public function boot()
{
    resolve(EngineManager::class)->extend('custom_scout_engine', function () {
        return new CustomScoutEngine;
    });
}

And finally, after your custom engine is registered, you can use it in the config/scout.php file.

'driver' => 'custom_scout_engine',

Conclusion

Today, we discussed how you can implement full-text search in Laravel using the Laravel Scout library. In the process, we went through the necessary installations and a real-world example to demonstrate it.