Why we killed elixir

First this is my opinion as CTO, I don't care about languages, I hate them all.

Here is my hatelist of languages I know enough to say I hate them, from most hated to least hated:

# most hatedjava perl javascriptelixir rubypython cpp objc clojurego dartc# least hated

All those languages have something to offer, when you decide to use one, you better know how it will bite you in the butt. Most overpromise and underdeliver, their philosophies work great on paper, but poorly in practice. By 'in practice' I mean a diverse team of people working on a non trivially sized project with different experiences, preferences and backgrounds.

When REKKI started the devs thought they are building a chat app, instead of ordering app, and they chose elixir, with phoenix to be able to trivially do things like 'someone is typing' in the chat. However what they didn't realize was that there is no way two chefs are in the app in the same time. You can read more about that here: https://dev.to/rekki/work-in-the-kitchen-4ifm

We moved from elixir to go around 1 year ago, it was pretty smooth, we just made few go services and moved endpoints that we needed. We made a super simple k8s deployment infra make build push deploy and you are good to go.

This enabled all frontend devs to write go, learn more sql, make simple endpoints for themselves, and change something if they need. All of a sudden each frontend dev became a full stack dev.

But elixir has so many powerful concepts, the genserver, hot code reloading, concurrency, its functional, phoenix is pretty popular and has great community, so why would we move away from it?

Let It Fail

First the "let it fail" thing, look, in our company things end up in a master database, for 99.9% of the things, you open a transaction, write your stuff and close it.

GenServer is great and all, but honestly for us its completely useless, it requires way too much discipline to not create inconsistencies in the data model, so if you do want to have things fail you have to actually handle the failure of a weak data model.

Lets look at this example:

User types "hello" message, this ends up in GenServer that sends it in rabbitmq, RMQ is great piece of software, if somethings goes in it is very unlikely it will disappear, except when there is no memory in one of the nodes because of a bug in another queue that the consumer was not consuming, so now lets examine this "let it fail" thing.

What exactly can fail?

Nothing really.. what about we just write the message in the database and return
ok or error to the user?

We had so many outages because the data model has to be very relaxed in order to support the "let it fail" thing, and so often we ended up with partial data in the database..

Checkout https://dev.to/rekki/mutation-is-life-boring-technology-11h0 for example

hot code reloading

For me this is complete myth, to build a robust hot code reloading deployment pipeline requires enormous amount of effort. To properly handoff and transform state properly..

Its ok if you want to replace one module or so, or for some prod debugging.

Performance

Runtime performance wise it is slow, about 2-3 times slower than go, and 4-5 than c, which is OK for most things, unless you want to rank a million items with some semi complicated formula, and at point you will have to scatter gather.

Compile time.. omg it is horrible, like early rust horrible.

Type Safety

We had to have huge amount of useless tests just to avoid having typos.. typos I repeat. Its like perl without strict. Running the linter is slower than flow. Of course we use typespec and etc, it just has the same problem as flow, things become any fast, and any spreads like a virus.

And worse you cant delete anything, first the tests will fail, and then even when you fix them you have no confidence your changes are good. Of course typesafe code does not guarantee things are ok, but I have seen elixir bugs that are just pathetic for year 2020.

Reading and Writing code

This is personal now, some people like it some dont, I wont make strong argument, for me reading clojure is 10x faster than reading elixir, but I also used clojure more.

The problem is it has steep learning curve, you cant get a js dev and ask him to change elixir code.

Scale

Do you really think the issues with our scale will be bound to elixir? Most companies will be fine by using bash+netcat webservers.

This is a joke obviously nobody should do that.

while true; do   echo -ne "HTTP/1.0 200 OK\r
Content-Length: 5\r
\r
hello" | nc -l -p 8080done

Concurrency

What concurrency? Most languages are ok with that, elixir is no better.

Finding developers

IDK, now its hard to find any kind of developers, but choosing niche language makes it even harder. Also people join just because of the tech, which is not good.

So since we wanted to unblock the frontend, and simplify the action-at-a-distance architecture, I had few reasonable choices, go and java amongst the top on the list, and since it is just easier to write ok-ish code in go than in java, I chose go.

So now we are a go shop, we have 20-30 services, each dev can make their own service and deploy it.

Things are pretty nice. I can say, after 1 year, it was a huge enabler and multiplier of people, I don't think we would've done half the things we did have we not switched, and the cost of switching was pretty small.

Go has its baggage as well.. but at least we can delete it.