300: Mozzarella Sticks & Knowledge Silos

About this Episode

The big "Three Oh Oh!" What a milestone for this podcast! Aside from celebrating that the show has made it this far, Chris gives some followup on some Inertia.js issues he had been having, and talks about open source licenses and legality and testing against external APIs. Steph has thoughts on mozzarella sticks and what makes good ones; particularly the cheese to bread ratio...

They then, together, answer a listener question re: knowledge silos:

Jan asked, "Our team (3 pairs) is currently working on two different projects due to that fact we are creating information silos. Now we are looking into ways how we can minimize those information silos. Do you have any ideas how we could achieve this?" With switching pairs they are unsure about it as it can be difficult for new pairs to get up to speed.

• inertia-rails thread safety
• Rails Cache-Control no-store fix

Transcript:

STEPH: I have no shame.

CHRIS: That's important in this industry.

STEPH: [laughs] Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Steph Viccari.

CHRIS: And I'm Chris Toomey.

STEPH: And together, we're here to share a bit of what we learned along the way. Hey, Chris. So today's an exciting day. It's a rather momentous day, at least in my world, because today is our 300th episode.

CHRIS: 300? That is incredible.

STEPH: That's an incredible amount of episodes. And it made me pause and reflect on how many episodes I have been a part of. And I've realized it's over 100. I think it's around 104 or something like that, and I can't believe it. Time flies when you're behind the mic.

CHRIS: Time does fly, yeah. So yeah, fully a third of these you've been involved in. I don't know what the number is. And I'm just so grateful to Derek Prior and Sage Griffin, who started this whole process. And then to Thom Obarski, who was the producer for so long, and Mandy Moore, who recently joined us and has been doing a wonderful job of carrying that forward and to you, Steph, because this has just been such a joy to work on. Yeah, it's just a joy to be on the show and to get to chat with you each week and share some things. And frankly, learn from folks writing in questions and sharing pointers with us, and it really is such a delight. And yeah, 300 is pretty momentous.

STEPH: The listener questions and feedback have undoubtedly been a highlight for me. That is one of the areas that I love the most. I love the questions. I also love when people provide helpful answers to us, and then they help us out in return and also, all the incredible guests that we've had on the show. It has been phenomenal. I'm also very thankful to have been part of this journey and appreciate everyone that has got us here today. I wonder what the fourth iteration of The Bike Shed looks like. I consider this the third iteration because the first iteration was Sage Griffin and Derek Prior. The second iteration was where you took over The Bike Shed, and then you were hosting a number of incredible guests on the show. And then the third iteration is the iteration that we're living, so I wonder what the fourth will look like.

CHRIS: Oh, that is an interesting question. Hopefully, you and I get to hang out for a good bit longer. But at some point, much like the Green Lantern, this will get passed on, and someone else will take up the mantle and tell some stories. But, yeah, hopefully, that's not too soon because I certainly enjoy hanging out with you.

STEPH: Oh, I agree. I certainly enjoy this, and I'm in no rush to leave The Bike Shed. But I think it's just fun thinking about the next people that will carry this journey forward.

CHRIS: And determine the color of The Shed.

STEPH: And determine. I mean, that is their right. As host and co-host, they get to determine the color of The Shed.

CHRIS: 300 episodes in, and we still haven't figured it out. So I guess we got to keep trying.

STEPH: Oh, I have. I already know what color it is.

CHRIS: Is it yellow?

STEPH: It's yellow.

CHRIS: Yeah. Okay. [laughs]

STEPH: I like how we said yellow at the same time, you know. [laughs]

CHRIS: I do, although I feel like it's wrong to have a color in mind, or at least I want to dig in and talk about it for a while just to be in keeping with the show, but...

STEPH: One must first argue before deciding and then argue again. But to not continue bikeshedding on The Bike Shed, what's new in your world?

CHRIS: My week has been good. Actually, I have two quick updates on various Inertia things that I've shared in previous weeks. So we can include a show notes link for the two different episodes where I talked about these respective things. But there was one weird issue that I ran into with Inertia where it could start clicking a button that would delete, was behaving weirdly and occasionally, intermittently; some of the responses would end up as a full HTML page response as opposed to the expected Inertia response. And there's a bunch of subtlety around this. I actually reported it as an issue to the Inertia team. And they very kindly pointed me to the HTTP semantics at place. So it's the difference between a 302 redirect and a 303 redirect. And so, in their code, they were correctly doing a 303. They were standards-compliant; everything was great. But for some reason, it was still misbehaving sort of randomly, and I could never pin it down. I ended up working around it and opting out of Inertia behavior for those endpoints. But my assumption was that something in my Rails Middleware Stack was behaving weirdly and occasionally overriding Inertia Rails' setting of the status. So Inertia Rails was saying, "303," which is a special version of redirect, and something else in the Rails Middleware Stack, was saying, "302, it will be fine."

Turns out, in retrospect, the Inertia Rails team has discovered that this was, in fact, a threading bug on their side. So it's not Inertia's fault. Inertia as a core concept and as a protocol was definitely doing the right thing. And the Inertia-Rails Middleware was attempting to do the right thing. But threads and concurrency got in the way, which I'll be honest, I don't deeply understand those concepts. So I was just like, oh okay, that sounds like a thing that could go wrong occasionally, which is exactly how I experienced it. But now they've made an update to the project, so that should be resolved in a deep way. But goes to show you threading and concurrency are really tricky to chase down.

STEPH: I appreciate that you're coming back to give us the conclusion to that issue because I remember talking about it, and you were still going off on a journey and finding out what's wrong, so that's super interesting. And yeah, threads and concurrency those are super easy, like cache invalidation and naming, that's right up there.

CHRIS: It's actually kind of funny. One of the issue threads where I wrote about it, someone followed up and asked if I'd come to any solution. And I said, "Oh, I've gone kind of this weird way, and I'm doing these things." But I shared a code sample, and I said, "Just to be clear, this is 100% about something Rails is doing and not Inertia, which remains a stellar project." And then, very shortly after that, someone from the Inertia-Rails team was like, "Ah, actually, I think it was us. Sorry about that, but we fixed it now." And I was like, "I still love you guys. This is great. You're doing a great job. [chuckles] You continue to push the envelope in a wonderful way." But it was a funny interaction where I was like, never shall I let the name be dragged through the mud. Whoops. Okay. Never mind.

STEPH: You're an excellent hype man for Inertia.

CHRIS: I try, I really try. I believe in it to my core. And actually, there's another one that this one's not really related to Inertia at all, although I've seen it discussed within the context of Inertia. And again, I think the Inertia team has done a really great job of responding and pointing to here are the HTTP semantics, and adhering to the standards, and the way that things should work. But this one has to do with the back button. When you're doing sequential forms or really any sort of form type thing, the browser will just pull from its back/forward cache, which is a local cache of the HTML of the page as it just had it. And I had come to the understanding that this was not something that I could workaround. This was not something that I could control. I had tried every combination of headers, at least I thought I had, in Rails to try and control this from the server-side because ideally, the server is the one who knows about when data is changing and things of that nature. The server should be able to inform the browser, "Hey, don't cache or store this page in any way, always revalidate it."

It turns out there was a bug in Rails that was improperly normalizing the Cache-Control header and always removing the no-store Cache-Control value. So there are like five different or a handful of possible values that can be set for that header for the Cache-Control header. And Rails has a bunch of internal logic that says, "Okay, if you've set this, then I'll put these two, but not that one." And they're just trying to manage it and do nice things on our behalf. But unfortunately, they were being a little overzealous in that normalization effort. And so they were dropping an important value, which is no-store. So now there's a PR opened in Rails, or I think it's actually been merged in at this point that will fix that and allow you to set that particular header value, which then should get the behavior of "Hey, browser, if I hit the back button, please go ask the server. Don't trust your local cache, “which is exactly what I want.

STEPH: Interesting. Wow. So that's two very helpful resolutions to some of those strange issues you were running into before.

CHRIS: Yeah, definitely. And actually, for that issue, in particular, it was a very kind Bike Shed listener; Alexei Vasiliev wrote in and shared some initial thoughts, pointed me in the direction of some things. In that case, I actually was like, "I don't think that's the case. I tried it." And he was like, "No, no, no, pretty sure." And he was definitely correct in this case and was very kind and gave me an example of code reproduction and all of those nice things. So I was able to chase this down and then eventually find the issue in Rails, which had been opened like eight days before. So I think for me, I just happened to run into a weird period of time where Rails was subtly broken around this behavior. And therefore, I determined that the world was broken when, in fact, it was just a tiny slice of Rails' history. But yes, thank you so much, Alexei, for writing in and pointing me in the right direction on that.

STEPH: The dream came true. We talk about some of our troubles and our strifes, and people respond and help us out.

CHRIS: That is the dream. But yeah, so those are some quick updates, not really about me, although tangentially, I got to go along for these rides, and it was fun. But what else is up in your world?

STEPH: Let's see. Well, I also have a small update that I can share. It's circling back to the conversation that we had talking about extracting an untrustworthy service to a new location. And at that time, I don't remember exactly the process I laid out. But at that time, it’s the idea that it is a bit untrustworthy, but we have some security in how this process works, and it is ideal that we move it to this other location. So let's just go ahead and move it wholesale, bugs and all to the new location. And then there, we will start to refine, and we'll start to improve the service. Well, the update is that we have realized that the untrustworthy service is untrustworthy enough that I'm actually working on improving it in its current place just to a certain extent that then it feels like we can move it to another location. There have been enough issues with it that it has taken my focus to continue patching those bugs and making sure everything is working appropriately. But now I'm in the space of where I'm like, goodness, I thought I knew this thing and now I'm realizing I don't. And so, I'm looking for ways to inform myself and the team when something isn't working when we think it is.

So to provide a bit of context, this service is sending a bunch of messages to other systems, and most of the time, that is working, but there are times that it's not. And when it's not working, it's silent about the fact that those messages aren't being sent, and it's very important that we send those messages. So what's been on my mind is looking for a way to then elevate myself and the team to say, "Hey, these are the number of messages that are being sent on average." And then suddenly, let's say it dropped by 50%, or maybe we typically send 98% successful messages, and we have a 2% failure rate, but suddenly we have a 50% failure rate, but looking for those metrics that I can capture and then alert the team if something is going wrong.

And one of the suggestions that was bubbled up by Chad Pytel, who's a developer, he's also founder and COO of thoughtbot, and we're working on the same project together. And he had highlighted that a previous project that he worked on used AWS specifically to leverage the idea of tracking how many successful messages are being sent, or perhaps in their particular project, it was focused on how many orders were being processed. That was important to know. And in our case, we could do a similar metric where we look to see are we still sending messages? Has the number dropped significantly lately so then we can be notified, and then we can escalate that to PagerDuty? So then we notify the team that something's going on.

I don't know the specific mechanics of how I'm going to implement that yet. So I will report back, but I'm excited to have something that's going to alert me for when things aren't working the way I expect versus waiting for then someone that's a customer to notice it and then get back to us. It's very in line with a number of the topics that you've brought to the show recently, talking about how we can measure more of the user's experience and be notified sooner versus waiting for a user to bump into an error and then they reach out and notify us.

CHRIS: I'm super interested to hear where you get with that because that's definitely an area that I've poked at but not dug into particularly deeply. I know there are a number of projects like StatsD is one of them. I think there are others in that space, but that's where you're sending metrics just out to some service, and then you can aggregate and graph. I've also done similar things with Papertrail; I want to say, where you can do a very specific search in the logs, and then within that, you can aggregate and graph and show things over time. So you can do a very simplified version of what you're describing to sort of visualize a rate of something over time. And then I think they might have some thresholding alerts.

But also, that's one of those super hard things to do because it turns out like Monday morning, a lot of emails get sent and then Friday afternoon, fewer, and then on the weekend, none. And so, there's going to be an inherent sort of fluctuation to the data. And so then what is normal? What does the baseline look like? And then how do you do anomalies around that? Because inherently, there's going to be noise in the data. And so is it a 10% band around the normal? And I'm just saying a lot of words now that I barely know the meaning of. But it's one of those things where it's like, oh yeah, just let me know if it's behaving abnormally. There's so much in that one little sentence. And it's one of the like; I love the fractal complexity of this space where every part of that sentence that I just said is like, oh, that's way more complicated than it sounds when you just say that word. So very interested to hear where you get with this. And this is also something that I'll probably be pushing on in my work in the near term. So maybe we can even compare notes, but as of now, I just have, I think, buzzword-level knowledge of it.

STEPH: Well, I love that phrasing fractal complexity because yes, that was also where my brain got hung up in starting to think about this process and like, well, what's normal? I don't actually know what normal looks like because I haven't been tracking this until now. So do I go back a week and say, "Okay, let's compare our average sent rate to in the past week and try to define normal in that timeframe?" And I think the answer, for now, is to do the smallest thing but also has the biggest impact, and that's to notify the team if messages just stop. That feels like the first, small step to take, and then we can fine-tune. Do we want to know if suddenly successful messages are being marked as a failure? We have an increase in failed messages versus successful messages. But I think the first iteration is just to know or to confirm that we are sending messages and send us an alert if suddenly we're not sending messages for...ooh, I just realized there's a complexity in that statement too. It's like, how long are we not sending messages for? Is it for an hour? Is it for a day?

CHRIS: I was going to ask.

[laughter]

STEPH: I just caught myself there. Yeah. I don't have an answer to that right now. I have to think about it, but there's an answer there. I will have to choose an answer.

CHRIS: You sure will. And then you'll probably have to tweak it over time. It's also one of those topics where false negatives and false positives are really easy to fall into where the system's alerting too often. And so people then start to ignore the alerts versus it's too cautious before it will send out an alert and, therefore, you're missing things and so finding that optimum level. It also might be different days of the week. Aah. [chuckles]

STEPH: Yeah, I think that's very true. It will be different for different days of the week. So I have a lot more to think about in regards to how we're going to report on this. But that still feels very much like something I want in the world because right now, it's a lot of spelunking and production consoles to find out what's going on with the data and making sure that it's going through. And that feels like the least favorable option as to the world that I want to live in.

Oh, on a completely unrelated topic, I saw an article that I'm very excited to read. And it's not related to technology at all, but it looks like a very delightful article that someone wrote and titled My 14-Hour Search for the End of TGI Friday's Endless Appetizers. And I haven't read it in-depth yet, but I just read the first bit, and it seems like it's going to be delightful. But I thought of you because we've had previous outtakes around mozzarella sticks. And you were very excited when you thought thoughtbot had mozzarella sticks, the actual fried kind versus just the healthier cheese stick kind. So this seems like a thing that you'd enjoy.

CHRIS: I feel like it may have even ended up in an episode, and we talked about mozzarella wedges and the ratio of surface area to volume.

STEPH: Yes.

CHRIS: I don't know if that made it into an episode or not, but we have definitely you and I discussed mozzarella sticks before. And I'm definitely intrigued by this article. I will add it to Instapaper immediately and then probably never read it again because Instapaper is where I put things to forget them. But maybe someday I'll sit down with a coffee and read things.

STEPH: I've heard you mention Instapaper before, and I've looked into it. And I don't know why, but it just hasn't stuck for me. So I always throw anything that I want to explore or something that is also critical for me to do. I use Todoist. I don't know if you're familiar with that app, but that's my go-to.

CHRIS: Well, I'm familiar with Todoist. I take a slight line between my to-do list, which I want to be as, I don't know, clean and tidy and only the things that I have to do versus for me, Instapaper is a list of when I get around to it when I've got those ten free minutes, which apparently don't exist in the world. But when I have them, this is the list of things that I can read. But I think I've heard this from a number of people of having a more integrated system that all the stuff's in the same place. I keep my to-dos in Trello, also as an aside, and I'm not super happy with that. How do you like Todoist? Is it bringing you joy?

STEPH: I really like Todoist. I find it is simple enough an interface that I'm not spending a lot of time customizing it or messing around with it. I can just go there and log the things that I want. I can create individual projects and spaces as well. So if I want to separate my personal to-do list from my work to-do list or if I have a project, that's a really nice feature as well. I think my only small complaint is if I'm writing a date or if I'm writing tomorrow, Todoist will try to do the smart thing and say, "Oh, I'm going to add a due date for you since you mentioned a date." And I'm like, no, no, no, I don't want a due date. I just want to mention the specific date because somehow it's relevant. And undoing that is sometimes a little tricky. But otherwise, I have found Todoist very helpful. I'm a big fan. Also, you and I are slightly different creatures in terms of how neat and tidy we keep our spaces. I think how we both manage our email inbox is a really good indicator of this where you are more organized than I am when it comes to emails. And so, our to-do list might be similar. I'd be interested to see if Todoist fits your needs or if it doesn't offer enough structure.

CHRIS: I almost certainly could make it work. And it's one of those things where I've actually settled on Trello, which is a very loose tool. And so I've been able to shape it sort of to what I want, but it doesn't really have that many true productivity-type features. It's just a loose board where I can drag around cards and move them through. And that's worked fine, or I've been able to talk myself into not trying to be as neat and tidy and intentional with my to-do list, which I think has been good overall. I've looked at Todoist in the past. And the thing that gives me pause sort of related to what you were talking about with the date things, but I get the idea, or I get the sense that Todoist really, from a fundamental philosophical approach, really wants things to have dates and to have priorities, and my thinking is not quite that. Like, there is a priority, but it's relative. So it's the order of things in a list, but it's not this is a one, and this is a two, and that's another two. I find that logic of like there are different tiers of importance doesn't really map to my world, nor do dates. Almost everything I do has no date, has no context. It's just like when I'm at the computer because that's the only place I ever am. So it's when I'm at the computer, it's all kind of important-ish. Nothing really has a date, but it should probably be done pretty soon. That sort of stuff doesn't quite map to what I see in Todoist. So I've always found a little bit of a mismatch between what I think I want and what Todoist, as far as I understand, provides. I know they added Kanban-type boards recently. So I think that might help with just visualizing workflow and being a little closer to Trello, which I'm familiar with. But I'm sort of on the search right now for another to-do list.

I like what you said about being able to separate the work and personal because that's definitely a thing that I would want, although there's always the added complexity of whatever tracking tool that we're using as a team at work and which things go into my list versus that list. And do I try and synchronize them in any way? And then I do what I do, which is I start to imagine this ridiculously complex, fully integrated, bi-directional sinking nonsense system where like, never mind. Stop it. Pen and paper, Trello. I don't know; you’ve lost your privileges, though. This is me talking to myself. I lose my privileges much like I'm not allowed to ever try Emacs. I have had a multi-year moratorium on exploring new productivity tools, but I think maybe, just maybe, now is the time to revisit that.

STEPH: If you ever disappear for a week or two, I'll know that you tried Emacs or something like that happened

CHRIS: [chuckles] My beard is three times longer when I come back, and I'm like, "All right. I figured some stuff out, though."

STEPH: I'm with you in regards to trying to bucket all of your to-do items as if it's a priority one, two, three. I am not good at that, and I'm always wrong. So I've also given up on that system. I would describe myself as a minimalist user. I'm using all the basic functionality. I'm not leveraging what a lot of stuff that Todoist probably can do for me. And so I have a very just flat list of things that I'd like to do. I do have a couple of projects because I do try to have that personal versus work, and maybe I have some other project that's on there as well. And then, in my mind, I try to avoid due dates unless it's really important. Although I say that if it's really important, it's going on my calendar too because I'm going to budget time for it or make sure that I don't forget it. But then each day then I go through that full list, and then I pick the things that need to be done that day or it's reasonable to get done that day, and then I kick everything to the next day. So that way, I'm always reevaluating a fresh list of what do I need to tackle? What's reasonable for today, and what can I punt on? And Josh Clayton said this to me before, and I really liked it in terms of punting on work because typically, when you're really busy, something's always going to drop. You're always going to push something to the next day. So then it's just figuring out what's going to bounce and what's going to break? So I'm always looking for what's going to break? And let's prioritize that for today to make sure it gets done. If it will bounce, then I'm going to kick it to the next day, and I can't see it until I'm going back through that full list again.

CHRIS: I really like that framing around you're going to have to drop things. That's just the nature of life. There's always more to do than there is time. So will it bounce, or will it break in that? And that framing around how to decide which things get moved out. Interestingly, I just looked up because I wanted to know does Todoist support snoozing things? Which is something that I use constantly in Trello and Gmail and basically everywhere else. I'm just like, nope, future me problem, future me problem, and I just keep pushing things into the future. But critically, I want them to be hidden until that time. And it sounds like Todoist; you can set a future due date, and then it'll show up in today. But again, that's sort of conflating how I think about productivity and whatnot.

Also, I found…this is a Reddit post that I'm looking at where I'm determining this. And there is the question, and then there's someone answered, but the answer is deleted. And then there's someone replying to that saying, "Wow, what a thoughtful response. Have you written this up anywhere else, like a blog post? You sound like an absolute pro." But the parent comment, which apparently was beautiful, and articulate, and well-written, has been deleted. And this is the sadness of the internet. So a really beautiful xkcd about the saddest thing you can see is you search for a question, and you find Stack Overflow from 10 years ago one person asking the question and no answers. And you've got one other person out there in the world who cares the same way you do, but you have no answers, and it's sad. But I'm just sad about the loss of information.

STEPH: That's so tragic, or that's a really pro troll move. And you leave a comment, and then below, you're like, “Wow, that was amazing. That was beautiful.” And then you delete your own previous comment. So then you're just tricking people into thinking there was an answer.

CHRIS: It does sound almost performative, especially the last line, "You sound like an absolute pro." So I could see that being the case. And you know what? I'm going to choose to believe that that's what it is because then I can sleep better at night. So thank you, Steph.

STEPH: Happy to help.

CHRIS: But I think we should probably move on to perhaps a listener question or something. But before we do that, I do want to ask if anyone out there has a to-do list that they're using and they love; I would love to hear about it. I think I'm familiar with most of them, but votes of confidence from the listeners of this show will certainly go a long way with me. Because I think you folks are all very smart people. I mean, you're listening here, so, obviously.

STEPH: Yes, obviously. This very deeply intellectual show about mozzarella sticks and the ratio of cheese to fried and what's the best.

CHRIS: It's an important question.

STEPH: It is an important question. I have strong feelings about it. That's why we've talked about it. [chuckles]

CHRIS: On this very serious show that we host.

STEPH: [chuckles] Yes, we have an awesome listener question that I'm really excited to dive into. But before we do, I have a quick git thing that I'd love to share. It's a tip that Dimitry, another thoughtboter, shared with me today that I think is just really nice and something that I have not used before. And it's specific to a workflow where if you need to grab a file from another branch or from another commit, and then if you want to bring it into your current branch. And there are a couple of ways to go about it. One of them is you can do git checkout main and then pass the file presuming the file that you want is in main and then you want to bring it to your current branch. And that will copy over the file to that exact location.

But if you wanted to grab a file that's on the main branch but then you want to port that file to a new location, then you can use git-show and do git show branch. So let's say you're bringing a file from main over to your current branch, so it would be git show main: and then pass to the file that you wish to copy, and then the greater than sign and the path to where you want that file to live. So you can grab that file and then stash it in a new location, and you can also do it for commits too. So if someone has pushed up a commit and you want to copy a particular file, say if you need to bring in some of their work into your branch, then you could also do git show commit, and then that colon, and then the path to the file. And then, if you wanted to move it to a new location, you can use that greater than sign and then the path to where you would like that file to live. So it's a nice combination of the git command of git show and then also shell redirection. So then, you can pipe that content from the file that you wish to copy over to the new location that you would like. And it's not something that I've reached for very often, but I find lately I've been in a mode where I'm trying harder and harder to stay within my terminal and not have to jump over to GitHub or to external UIs if I can. And so this just feels like a nice additional tool where then I can use this one more thing where I don't have to either...I guess it's small. I could check out main locally. But even with this way, I don't have to switch branches, grab something, and bring it over, or I don't have to go to GitHub and then look for something. It feels like a nice way that then I could grab that file locally and bring it over to my branch.

CHRIS: That's a nice combination of tips there. Like you said, a bunch of different pieces at play, but that is definitely a super useful thing. It's one of those that I've not gotten that into muscle memory yet or even close to muscle memory. Git is complicated in terms of the interface that it provides, at least at the command line. I've been trying to make sense of it all and then trying to find what are the useful workflows that I want to build? Because you can do anything, and you can do most things in five different ways. And so finding that set that you do want to know deeply but then also getting that committed into your hands, not even into your head, is the thing that I strive for. But that particular one is one that I struggle with every single time. So especially, I think you broke that down really nicely, so it makes sense.

There's a corollary in Fugitive for any Vim users out there. There's a Gread command, so it's capital G-re-a-d. And then after that, it takes some identifier, and I've never gotten the identifier right. But as you just described it, it's the same as the git show sequence. So it's a commit or a branch name, colon, and then the file path that you want. And then, in Vim, you can use % to reference the current file. So I've tried really hard to teach my brain Gread main :%, and somehow, my brain doesn't want to remember that ridiculous sequence of characters. So, only in this moment am I like, oh, it all kind of fits together.

STEPH: Oh, that's nice. I am a Vim Fugitive user, but I didn't know that one. And I'm with you; I rarely remember all these off the top of my head unless I've done them like a hundred times, and it finally starts to sink in. So I always have a cheat sheet, or since we were talking about tooling earlier, I use Notion to capture tidbits for myself. So this is a place where I would probably stash in a web development folder that I have. And it's just a tip to my future self as to like, hey, remember when you were trying to do that thing, and then you had to look it up and figure it out? Well, here's how you did it, so then I can revisit it in the future.

CHRIS: I thought a number of times about introducing a flashcard system to revisit these sorts of things. Gary Bernhardt, who I had on a while back now, is building a platform that does this essentially for TypeScript and regular expressions in JavaScript arrays and a bunch of different topics. But it's got built into it the idea of spaced repetition, so you review a thing and then three days later, you review it again and then seven days later, and then ten. And there's a particular sequence to it, but it helps you to really internalize that knowledge. I've never gotten to the level of going to that, but I like that idea of being purposeful and trying to commit some things to memory because having them at your hands and being able to stay, like you said, in the terminal and closer to the work and not having to break out of the context, I do find a lot of value in that. But it does take some effort to build that up. So I've never quite gotten to that flashcard system myself.

STEPH: Yeah, that's interesting. I think I have mixed feelings about it because, on one hand, it is nice to commit some things to memory. And on the other hand, I'm totally cool with having a way to organize stuff so I can easily search it and find it later and not use up memory space for something that I don't use that often that then I just can't commit it. So I could definitely see it being useful. But I'm also okay with just having a nice way to search for it.

But pivoting a bit and circling back to the listener question that you alluded to earlier, we have a listener question from Jen and Jen wrote in about knowledge silos across different projects. Specifically, Jen wrote in "Hello, Steph and Chris, first of all, I want to say that I love to listen to your podcast for multiple years now." That's awesome. Thank you, Jen. "I like how you both share things along your week and fill the discussion with so many useful things and findings. Our team, which consists of three pairs, is currently working on two different projects. And due to that fact, we are creating information silos. Now we are looking into ways into how we can minimize those information silos. And do you have any ideas for how we can achieve this? Some additional context, switching pairs we're unsure about as this will be difficult for the new person to get up to speed. And currently, we are thinking about having a mob review session. But of course, with those, you only get a limited overview." All right. Well, thank you, Jen, for the question. I'm excited for knowledge silos because, I'll be honest, I am guilty of this one right now. I am a bit of a knowledge silo on my current project if we're telling our truths here on the show today.

CHRIS: Steph, I thought I knew you.

STEPH: You know, I'm full of surprises.

CHRIS: Aren't we all at various times? This really does feel like one of those core things that I associate with you, though. So it is interesting. But it's so easy to fall into this space. I think without purposeful, intentional effort, this is the natural way things will trend. It's so much easier for the person who understands a portion of a system or an entire system to take on the next piece of work for that system. And I think we can probably offer some specific advice. But to talk about it more generally, Jen, I think you've found yourself in the pretty common position of there isn't a great answer here. There's going to have to be an investment of some amount of effort; some potentially decreased productivity for a period of time in order to get out of the situation that you're in. But that's just the name of the game. So if we name it as that, and we say that, then the question becomes how much effort do we need to put towards that, and what are the different ways that we can do it?

So to go through the two that you listed, mob review sessions, I think can be a great way to give an introduction to a project, but I think they'll very quickly taper off in my experience. So I think it's a great way, especially if you're going to do any more formal things after that; a mob review or even a mob overview of the system is a great way to introduce new folks into it. But then from there, I personally would think that if you are feeling pain around the knowledge silos or even if you're not, because frankly, knowledge silos can very quickly become a major problem, say if someone needs to...if someone happens to leave the company or if someone needs to take some time off, anything of that nature, this is one of those things that can be fine until it's not, and then it's not in a very serious way, and that's the wrong time to try and resolve it. So I would very much be in favor of more purposeful things.

As you described, switching pairs is an interesting one. I think that's a cost you're probably going to have to pay. I am interested; the way you're talking about it, it sounds like your teams are paired up consistently, so you're working exclusively in those pairs, which frankly is a really interesting thing. I think it was the previous episode where Steph and I talked about agile and particularly 100% pairing, and that's a pretty intense idea. It also does potentially lean towards this. Now, each of those groups of people, each of those pairs is collectively aware of the same subset of the application. But now, if you were to split that up and you have six individuals that pair in varying sets across the different projects, you have this sort of Venn diagram tapestry of knowledge of the different systems and the subsets and the features. And for that reason, I actually would probably question, at least if I'm correctly interpreting it, that you have three consistent pairs; maybe you shuffle that up. Maybe that's a practice that should be unwound. And now the pair should rotate on a daily basis or something to that effect. But overall, I think this is a cost you're going to have to pay but will pay off longer term. And it's definitely worth doing in my mind. But yeah, that's some high-level thoughts. What do you think, Steph?

STEPH: I agree with all of those sentiments very much. And as you're talking about the cost and investing in the team, I think that's very true and something that needs to be done. The fact that they're working in pairs is already reducing knowledge silos because you at least have another person. Because I have been part of teams where there's one person that is that knowledge silo. So at least here, we already have two people that are aware of how code works and then why code was implemented in a certain way. So then, to categorize how painful that knowledge silo is or how risky that knowledge silo is, I think there are really two ends of the spectrum. And on one side, there's that example that you alluded to a little bit ago about isolating one developer on a project for six months, and they have minimal code reviews. And then suddenly that person leaves, and that's the hardest silo to then rectify. And it will probably be a lesson that stings enough that hopefully it won't be repeated where someone gets that isolated and then others have to figure out what was going on while that person was working on something independently.

And then on the other side of that spectrum is you need to take some time to explore and understand a portion of the application that you haven't worked on before, or perhaps it's you need to understand how to work with an internal API. And stuff on that side of the spectrum feels more addressable with documentation and also mob reviews. And maybe there are also demos as well because a lot of the knowledge that goes into building a product may not be specific to the code, but it's more why was this done, and why was it built, and why did we go this way? And that feels more addressable with documentation, with commit messages, with those mob review sessions, and also with demos where then you can show the high-level functionality of a feature that's being implemented. So then, even if everyone else on the team doesn't have the technical knowledge as to how it was built, they'll have more of the user context, and the product context as this is a feature that we built, and this is why it's useful to the world. I find a lot of that knowledge is what's harder to capture because then you'll find a feature and wonder who uses this and how is it in use? And that stuff is harder to backtrack.

Circling back to something that Jen caught out in their question, highlighting that it takes time for someone to get up to speed. That's a really interesting one for me because it goes back to the idea of wanting to know well; what’s difficult? Not specifically what is difficult, but let's define difficult and what's a reasonable level of difficultness because onboarding to any applications or onto a new section of code is always going to take some time to process and understand. But what's an acceptable timeline in which someone can onboard and be productive? There's something that I've heard from someone at thoughtbot. I don't have the exact context to quote them directly. If I find it, then I'll be sure to add it to the show notes. And they shared that another company is measuring this difficulty of onboarding by they take the person's first starting date, and then they track to see when that person has merged in 10 PRs because they are looking to see how long it took for that person to get up and running to then feel comfortable, to then make some contributions.

Often, your first couple of PRs might be something that's less challenging. It might be something that's updating the README because you are going through that onboarding process. And that's a great time to then reevaluate how clear the instructions are. But by the time you get to the 10th PR, you've probably addressed something that's a bit meatier and impactful to the product. And then they use that as a metric to then calculate okay; how well are we doing? Is it a month? Is it six months until someone gets there? How complicated is the application is another way that you could look at that metric to say, "Well, if it takes people a very long time to get there, maybe it's because of the codebase versus processes." And I really like that thinking of we have knowledge silos; let’s think about where it's actually hurting us. And then, if we think it's specific to the onboarding process where that part is painful, then let's break down how we can measure how difficult it is, and then look for ways to improve it but then also track that improvement.

CHRIS: Well, I like that idea of trying to quantify and measure onboarding. I've heard a lot of organizations having like, "We want you to ship a PR on your first day," that's a meaningful thing. But obviously, that first one will probably be pretty small, and it's sort of getting that first one out of the way, if anything. But it's not truly representative of someone being able to comfortably work within the repo, but ten, that starts to feel like a real number. And I do like quantifying it. More generally, I'm intrigued. Metrics around developer productivity is such a difficult thing to pin down. And it can, I think, become really complicated, especially if you're looking at individuals and trying to say, "Well, you had four PRs, but you had two PRs," and comparing individuals. But I do really like the idea of more aggregate stats of on average; right now last month, we were doing 1.2 PRs per week per developer, and now we're down 2.7 PRs per week per developer, something like that, and seeing that looks like something that we might want to address. Are there fundamental things that are happening that are causing development to slow down? Are we doing bigger PRs, et cetera? And starting to look at that, but try and have a metric to keep an eye on that. So I'm super intrigued by that and then again, more specifically to the onboarding one that you were talking about there.

Actually, popping up a slightly higher level, though, I think both you and I sort of jumped into this conversation as, like, yes, knowledge silos got to fix those, that's a problem. And I do feel that way. This is a topic that I feel pretty strongly about and pretty clearly about that knowledge silos are the natural state that things fall to, and it's not a good thing, and we want to avoid it. But it is important to ask the question of who is deeming this to be a problem and for what reasons? And we had a good conversation two episodes back in response to a different listener question about consulting versus building product. And I feel like, with this, we can almost go up to the consulting level of this can be a problem, but it also maybe isn't. Or, who believes it's a problem? Is it management thinking, "Oh no, when that person went on vacation, suddenly everything ground to a halt? This is a problem, and we need to resolve that." Or is it the development team themselves saying, "Hey, we feel like we're a bit siloed here, and that's a problem we're recognizing," but they don't have buy-in from management. Or worst case management saying, "This is a problem, but you get no time to resolve it." As long as everyone's in agreement of the potential benefits and aligned to this is a thing that we would want to improve, and then also aligned to there will be a cost to resolving it, that it's not free to try and unwind this siloing of knowledge, then I think everything can be great. But any mismatch at sort of any level of that either on the cost or the benefit side can be problematic. And so getting to the point where you've had a clear conversation that defines this and gets everyone to come to an idea of yes, we think it's a problem, and yes, we want to put in the effort to resolve it, then I think you can move forward and tackle any number of different approaches. But I think you have to start from that conversation.

STEPH: I love asking that question of how has this manifested into a problem or a concern? Because you just highlighted a really great example where if it's only a concern because someone was on vacation and the team couldn't respond to a customer request or couldn't respond to an outage, then there are different ways to address that. So documentation may not be the best way to help out with that. That's probably a pairing session. So then someone can respond quickly to an outage versus you don't want to say, "Okay, here's a couple of pages of documentation," and then have that developer go on vacation again, and then there's an outage, and you're trying to read through those pages to figure out what's wrong. So figuring out the right approach based on the pain that's being felt feels like a really great way to go about this. Because frankly, breaking down a knowledge silo is always going to have a cost. So you want to make sure that you're being as cost-efficient as possible with your approach and then addressing the root concerns and making everybody's lives better.

Because I do think there's some knowledge silo that's appropriate. And I think silo may be the wrong word, but someone who is more skilled or an expert in the area or has more context for a particular area of the application. Because applications can get so large that not everyone's going to know everything and context switching between all of those can be really challenging. So I think it's very natural that you're going to have different people that you go to around a certain feature. If there is some lofty feature around search and you know a particular person that has worked on it for a while, then you go to them, and that feels like an appropriate level of knowledge that someone has obtained. And I wouldn't classify that as a silo at that point. But then if you do get to the point where that person went on vacation and then search broke, then you can start to revisit okay, maybe this person does have too much context, and then we can offload some of that context to someone else.

CHRIS: There was a phrase I used earlier of like a patchwork quilt, but I think that's not quite the right image. There's an image in my mind of little islands of color that are fully separated; that’s bad. And then there's a version of more like a Venn diagram overlap where each of the colors sort of bleeds into the other ones, and I think that's good. But then the perfect overlap where it's just one big blob of brown because all the colors are the same, that's bad. And I think that's what you're highlighting is like, you don't want to go to that. You don't need the perfect overlap of everyone having a complete shared knowledge set. I'm trying to make word pictures over internet radio. So it's probably not going great, but it's something to that. Like, there is an optimization here, and I think the way to find that is by starting from what are the pain points? What are we feeling that is less than optimal? And then coming up with solutions that directly address those pain points, not generically try and target like knowledge silos bad. And retros are a perfect way to do that. So if you listen to our previous episode where we talk about the virtues of retros and other agile philosophies...This is great. I feel really good about being able to reference previous episodes. I think we've talked about good stuff in the previous episodes.

STEPH: You've been on fire with this episode. I think you've referenced at least two, three episodes at this point. [chuckles]

CHRIS: Yeah. Wow. Well, I mean, we're at 300 now, so we've got plenty to go back to.

[laughter]

STEPH: We've got plenty of content to reference. I think you and I do have an advantage here based on our experience where we have had to join a number of projects. And then we know our time with that project is very determined, and we want to make sure that we don't take any knowledge with us. So something that you and I have acquired as a skill is seeking knowledge when we first join a project and asking a lot of questions around how the application works and then understanding more about the intent of different features, and then knowing where to dive into a codebase to then make fruitful contributions. And I think there's a similar approach that can be taken when trying to break down a knowledge silo is a person who is that silo may be in a spot where they're having trouble communicating all that information and then dispersing it to others. So then us, as their teammates, can go to them and try to ask those types of questions to then help ourselves level up and then recognize areas that don't feel documented. And maybe it's adding documentation, maybe it's adding tests, or maybe it's doing a demo, maybe it's recording something about the feature and then sharing that with the team. But then you can be an advocate for that person who is in a silo position to then help them share that knowledge because they may be too far down that path where they don't recognize what they know, and other people don't. I don't know if that's directly related to being a knowledge silo but just an additional way to approach helping breaking down when you recognize that a silo does exist and looking for ways to then help that person communicate and distribute their knowledge.

CHRIS: Yeah, I think you're describing a distinction between a push versus a pull. It could be incumbent upon the person who has the knowledge to try and push it out to the team. But often, they're going to be perhaps a more senior person. They've got code review to do. They've got other meetings, and planning, and things, and they just may not have the time. But is there a way that other team members can proactively pull that information from them and help them find the moments that will clarify that? So, yeah, broadly, as a team, let's rally around the desilofication of the whole adventure.

STEPH: That's exactly what I was going for is that push versus pull mentality and how we can break down the silo from both sides. So thank you, Jen, for that wonderful question. I hope we gave you some helpful ideas and suggestions around addressing a silo and then also identifying the pains that you're feeling so that way you can find the most cost-effective approach. But on that note, shall we wrap up?

CHRIS: Let's wrap up.

STEPH: The show notes for this episode can be found at bikeshed.fm.

CHRIS: This show is produced and edited by Mandy Moore.

STEPH: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or a review in iTunes as it helps other people find the show.

CHRIS: If you have any feedback for this or any of our other episodes, you can reach us @bikeshed on Twitter. And I'm @christoomey.

STEPH: And I’m @SViccari.

CHRIS: Or you can email us at hosts@bikeshed.fm.

STEPH: Thanks so much for listening to The Bike Shed, and we'll see you next week.

All: Byeeeeeeeee!

Announcer: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success._

Support The Bike Shed