Reducing rerenders in nested/recursive components?

Codesandbox

I'm trying to decide how to port an old pure-js Tree library to React. Because tree components are recursive, no matter my approach I'm seeing rerenders (react devtools profiler w/"record why each component rendered" enabled) because state has changed and needs to be passed to children.

I've been able to avoid rerendering childless nodes by only passing state setters, but since state has to be passed to children, any node with children renders. So if I click a node to select it, all nodes with children rerender.

I'll need to eventually handle more than just an isSelected state, this is just a simple demo, but I can't figure out a good way to do this.

I could write my own logic to determine if a parent node needs to render but it would need know if any child nodes have changed state. A recursive check is bad, so I was thinking of setting some kind of isDirty flag on parent nodes, but being fairly new to react I don't know if there are better ways.

I'm hoping to avoid third party libraries as I'd like for this to be a public library eventually.