Release 2.7.0

I never planned for these late night posts, yet, it seems to becoming a trend. It's always on Friday nights when something pops up that I just want to get over with before I start enjoying the weekend and pretend Telescope doesn't exist. But really though, we all know students don't have real weekends.

So, to ensure this doesn't turn from a late night post to an extremely early morning blog, I better get started.

Release 2.7

This week was release week and we're now on 2.7.0! This release was really busy with over 80 issues and PRs that were addressed. Prof has a blog post on this with more detail.

At the time of this post, I think there's a scheduled 2.7.1 release some time during the weekend. To sum it up, something broke, its now fixed, just waiting for a mini release.

I managed to sneak into the later parts of the initial release meeting to see the Prof, our Sheriffs (Tue and Francesco), and some other classmates knocking their heads to get things working again. The causes and fixes are out of my league, but maybe they will be revealed in the blog posts on Telescope.

pnpm Problems

All throughout the meeting, I had my fingers crossed that I didn't cause any of the breaks. Which, shouldn't even be a concern cause I didn't do all that much this release. I had my advanced search tests merged, so yay! But, they were stalled mostly because of issues with getting the right Satellite version.

Somehow running pnpm install wouldn't update Satellite to its latest version. I had thought the solution was to change the version to "latest" in package.json, and even closed the issue with a PR, thinking that was the right fix. But nope, it was just an illusion. Before I even got to researching how to automatically get Satellite updates, the problem showed up again.

It seems that the pnpm-lock.yaml won't update if nothing in the package.json was changed. Currently, the fix is to manually update with pnpm update -r --latest @senecacdot/satellite.

Coming up next

Here are some of the issues that I'm currently assigned to.

Documentation

One for Auth docs, because I joined the meeting on Supabase to get an idea of what it's about. Originally I had figured I needed to know Supabase since it might eventually take over ElasticSearch in Telescope. But it actually seems to be really neat and I'm still getting to know it and exploring the possibility of using it in my personal project.

Another one for translating the About page. During our triage meeting we were exploring the possibility of translations. I think if eventually set up correctly, it might create some easy good-first-issues to attract future contributors.

pnpm and maybe CI

As mentioned above, currently, to get the latest Satellite version, we do things manually. The issue is to explore the possibility of doing it automatically. I am not too sure if this is possible, since any pnpm-lock.yaml change requires merging a commit. So, maybe we can do it semi-automatically, kind of how renovate does it for the other dependencies. I'll have to explore more on this and also discuss if this is still needed.

Docker

I kept myself on the team for creating our own Docker registry. I am still catching up to the cloud computing classes, but since I made all that effort to "sneak" in, I might as well find a place to get handsy with it.

More ElasticSearch

This comes in three parts, fixing a bug, refactoring and updating to ES 8.0, getting in some Autocomplete.

Part 1: Dealing with ElasticSearch errors

This late night post is a result of working on the Satellite related issue. Though, I want to emphasize how nice it is to have tests so I sort of know what to look for.

It started with Prof running into a crash related with our Search code. Here's where the crash happened when we try to createError (from http-errors)

// query.jsrouter.get('/', validateQuery, async (req, res, next) => {  try {    const { text, filter, page, perPage } = req.query;    res.send(await search(text, filter, page, perPage));  } catch (error) {    console.log({ error }, 'query error');    next(createError(503, error));   // <-- crash is here  }});

It could've been our problem, ElasticSearch's, or http-errors'. I'll give the conclusion here, so you could skip the next parts if it is too lengthy. "ElasticSearch" has their own custom errors that couldn't be recognized by "http-errors", which we tried to pass in as a regular Error in "our" code. I'd say, it's probably mostly our fault for not having tests to catch it sooner.

I figured it out thanks to my own tests that I landed.

I did what Prof had suggested, and added a test in Satellite with Error objects. It passed, so I was a bit more confident that it was our fault, the way I assumed and written last week.

The fact that Search seemingly wasn't handling errors correctly was still bothering me. Therefore, I tried to refactor our Search code to handle errors, and also wrote tests to check the results. But, the error was still occurring. Eventually, it gave me the idea to add tests with ElasticSearch errors, and the tests results confirmed it was indeed related to ElasticSearch's fancy errors.

test('should fail when directly creating Error from ElasticSearch error object', () => {    // { errors } is imported from ElasticSearch    const elasticError = new errors.ResponseError({      body: { error: 'testing ElasticSearch Error' },      statusCode: 404,    });    try {      createError(503, elasticError);    } catch (err) {      expect(err instanceof TypeError).toBe(true);      expect(err.message).toBe(        'Cannot set property statusCode of [object Object] which has only a getter'      );    }  });

The cause might be from ElasticSearch's change with errors in 7.16. It was also probably not noticed much because we don't get that many errors from ElasticSearch?

Anyways. As a follow up, Prof asked to explore the possibility of handling this internally within Satellite, since we export createError from there.

I was initially worried about createError being immutable from being imported, and that we might have to change the function name. Luckily, Prof came to the rescue.

I am so glad I am still at school so I can still ask questions like this that make me seem dumb. But I'm really glad I know this trick now.

Following Prof's tip, I have made a PR to take care of the ElasticSearch issue internally. It might have been a bad time now that I think of it, since its the weekend and Renovate is going to try to make many dependency update PRs. Nonetheless, I hope for my PR, after reviews and tweaks, will be able to fix the issue.

Part 2: Updating to ES 8.0

Rumor has it that every time we update ElasticSearch, Telescope breaks. I've been reading the release notes, and I've already came up with a list of features that would possibly break our code.

• Dropping callback API
• Removing body key from request
• The returned value of API calls is the body and not the http related keys

Some other things to note in the API changes (will have to scroll down all the way to Search for the whole list).

• Fix range query on date fields for number inputs #63692 (issue: #63680)
• Remove type query #47207 (issue: #41059)

Part 3: Autocomplete

I've been trying to read more on ElasticSearch, and I realized that our Search only uses a teeny tip of what ES has to offer. I also found, to my surprise, that even GitHub uses ElasticSearch.

Up till now, I've mostly been debugging, so I'm hoping I can actually get some more ES features in before Supabase takes over. Autocomplete could be a nice place to start out.

I think we might need the Parser service in first so I can get the indexes in. I still need to get reviews in for that. Also, before I can tackle getting in new features, we have to get parts 1 and 2 completed. Furthermore, if I had understood correctly, Amasia, who's been working on the search bar redesign, has also changed some of the Search code, and might have a PR up soon as well.

Conclusion

At this point of writing, when it might actually be categorized as an early morning post, my head is just thinking, "Oh gosh, there's so much to do", and yelling at me to sleep. But I know really, that I'm enjoying the work that I've done, and might still get to do for this course.

On a side note. Just last night I had thought maybe my Hacktoberfest T-shirt is forever lost in the abyss. Just checking now though, it finally has updates. So, maybe there's still hope.