Step one in making a new repo is disabling Dependabot. Automate intentionally or get pwned your choice.
Also, is automated version bumps really such a good thing? Many times I have wasted hours tracking down a bug that was introduced by bumping library. Sometimes only the patch version of the library is different so it shouldn't be breaking anything... but it does! It is so much better to update intentionally, test, deploy. Though this does assume you have a modest number of dependencies which pretty much excludes any kind of server-side javascript project.
I don’t disagree about automating intentionally, but it’s worth noting that Dependabot isn’t enabled by default: you have to explicitly configure it.
(The larger problem here isn’t even Dependabot per se, since all Dependabot does is fire PRs off. The problem is that people then try to automate the merging of those PRs, and end up shooting themselves in the foot with GHA’s more general footguns. It also doesn’t help that, until recently, GitHub’s documentation recommended using these kinds of dangerous triggers for automating Dependabot.)
> Really? Dependabot runs on a number of my repositories without my having consciously enabled it.
I've never experienced this. Do you have a `.github/dependabot.yml` file in your repository? That's how it's enabled.
(GitHub has muddied the water here a bit by having two related but distinct things with the same name: there's "Dependabot" the subject of this post, and then there's "Dependabot security updates" which are documented separately and appear to operate on a different cycle[1]. I don't know if this latter one is enabled by default or not, but the "normal" one is definitely disabled until you configure it.)
My understanding, and it may be wrong, is you may be grandfathered in to an ancient Personal, Public Repo opt-out from a brief window of time just after GitHub was excited to announce the first/earliest version of Dependabot and was hoping it would clean up some Open Source supply chain attacks and just before GitHub realized Dependabot was a useful thing to charge people an upcharge on (now under the umbrella known as GitHub Advanced Security). I believe that GitHub auto-opted in a lot of personal accounts with "significant" Public repos (anything with a bunch of forks/stars, or a package identifier visible in the dependency graphs of the Ruby or npm ecosystems, or any of the things that awarded "badges" like Mars Rover badge or the Artic Vault badge). There's a page buried in your Personal Account Settings to turn off that ancient Dependabot option. (I'm on a work machine without access to my personal account at the moment or I'd directly tell you where to find it.)
I'm at a loss to explain that! My only other guess is that you might have enabled Dependabot at some point further back in history, when it was a third-party integration and directly owned by or integrated into GitHub.
Do you have a Dependbot entry in your account/org-level applications?
Okay, I have no idea then. I guess perhaps at one point Dependabot was enabled by default for some people, although that strikes me as a bad idea and I can only assume they've disabled it since then, since I haven't seen this on any new repository I've made.
> After you finish apologizing to all your users for the recent breakages, you decide to make some performance improvements. Everyone loves faster code, right? This is a safe thing to do and surely nothing could go wrong.
> The new app now calculates the yearly tax summary almost instantaneously. It’s a huge improvement over the previous version, which used to take several seconds. You ship it.
> …and oops, one of your biggest partners has called you to complain. You’ve broken their website, which embeds your app as part of their business management suite. It turns out their code expected your calculation to take at least 5 seconds. Now that it’s faster, users encounter lots of errors and results that don’t make any sense.
> In frustration, you quit your job and return to the construction industry. At least here, no one expects a house upgrade without disruption.
How do you scale this besides keeping the dep list short? Are you reading every item or just scanning for words like "deprecated" or "breaking change"?
How do you prevent exposing yourself to supply chain attacks like the tj-actions/changed-files one [0] if you don't?
I get your question regarding scaling, but that's the job: you can choose to outsource code to 3rd-party libraries, and eternal vigilance is the trade-off.
Assume your 3rd-party dependencies will try to attack you at some point: they could be malicious; they could be hacked; they could be issued a secret court order; they could be corrupted; they could be beaten up until they pushed a change.
Unless you have some sort of contract or other legal protection and feel comfortable enforcing them, behave accordingly.
That's a judgement call. It would be too much to review all code change of all dependencies unfortunately.
The corollary of reviewing all code on all dependency updates is you should review all code or the new deps you add, including the transformation by build processes that might mean what is in the package manager might be different and same for all transitive dependencies.
> Sometimes only the patch version of the library is different so it shouldn't be breaking anything... but it does!
Still have flashbacks from that one time when some dependency in our Go project dropped support for go1.18 in its patch version update, and we almost couldn't rebuild the project before the Friday evening. Because obviously /s being literally unable to build the dependency is a backwards-compatible change.
> Sometimes only the patch version of the library is different so it shouldn't be breaking anything... but it does!
OT: Semantic versioning’s major flaw is that is the presumption that the entire chain of package maintainers is extremely diligent about correctly bumping their version numbers. There have certainly been a few projects that are very good about this, but the vast majority are not.
The only solution to this that I know of is to test: Manually exercise the features with the dependencies, write automated tests that check your critical needs from a dependency, or identify and run the tests from the dependency’s test suite that cover your use cases. (Also: Contribute tests that you want in the suite!)
I'm going through SOC2 right now, and it is essentially requiring it to be enabled. I'll just do the minimal to pass, but it isn't something you can just disable in some cases.
FWIW: this might be a suggestion of the specific audit team you’re working with or a requirement of one of the “follow our playbook and you’ll pass” vendors if you’re using one of those, but the SOC 2 on its own doesn’t really impose specific technical feature/control requirements like this.
I don’t have the exact exam language in front of me right now but the requirement would be something like “you have some process for learning about, assessing, and mitigating vulnerabilities in software dependencies that you use”.
Enabling an automated scan and version bump tool like dependabot is a common and easy way to prove your organization has those capabilities. But you could implement whatever process you want here and prove that you do it on the schedule you say you do in order to satisfy the audit requirement.
True on all counts. But the lowest effort is "just turn off dependabot", which is what I suspect most of the people trying to get past SOC2 will do (like myself).
> Also, is automated version bumps really such a good thing?
Depends. Do you want to persist the belief that software requires constant maintenance because it's constantly changing? Then yes: automate your version bumps and do it as often as possible.
If you want software to be stable then only update versions when you have a bug.
Curious if auto-merge philosophy changes between libraries and applications. The library definitely has a larger user base to break and a wider matrix of use-cases. IMO, auto-merge is more palatable for an application – do you agree? Especially when you're under SOC2/FedRAMP/etc.
Work has been fiddling around with Dependabot and force-enabling it everywhere (not auto-merging, just having it generating the PRs)... my feedback was that it is built on the presumption that All Updates Are Automatically Good, but this is transparently a false statement. In fact, Dependabot, by being so fast and automatic, may actually raise the probability of some project getting malicious code injected into it! Consider the timeline of malicious code injection:
1. Malicious code is injected into some project.
2. People have a chance to pick it up and put it into their code.
3. The malicious code is found, publicized, and people react.
The faster you act after step 1, the more chance you'll have time to put it into your system before the world reaches step 3. Dependabot maximizes the speed of reaction after step one. If I'm doing things somewhat more manually then I'm much more likely to experience the world finding out about a corrupted dependency before I start incorporating it.
Now, just typing it out it may sound like I'm more freaked out than I actually am. While supply-chain attacks are a problem, they are getting worse, and they will continue to get worse, they are also still an exotic situation bubbling on the fringe of my awareness, as opposed to something I'm encountering regularly. For a reasonable project the most likely outcome is that dependabot enhancing this exposure window will still not have any actual real-world impact, and I'm aware of that. However, where this is relevant is if you are thinking of Dependabot and its workflow as a way of managing security risk, because you imagine updates as likely carrying security improvements, and that's your primary purpose for using it (as opposed to other uses, such as, your system slowly falling behind in dependencies until it calcifies and can't be updated without a huge degree of effort, a perfectly reasonable threat and Dependabot is a sensible response to that), then you also need to consider the ways in which is may actually increase your vulnerability to threats like supply-chain attacks.
And of course, projects do not start out with all their vulnerabilities on day one and then monotonically remove them. Many vulnerabilities are introduced later. For each such vulnerability, there is a first release that includes them, and for which treating that update as if it was just a Good Thing was in fact not true, and anyone who pushed it in as quickly as possible made a mistake. Unfortunately, sometimes hard problems are just hard problems.
Though I have wondered about the idea of programming something like Dependabot, but telling it, hey, tell me about known CVEs and security releases, but otherwise, let things cook for 6 months before automatically building a PR for me to update. That would radically reduce this risk I'm outlining here.
(In fact, after pondering, I'm kind of reminded of how Debian and a lot of Linux distros work, with their staged Cutting Edge versus Testing versus Stable versus Long Term Support. Dependabot sort of builds in the presumption that you want that Cutting Edge level of updates... but in many cases, no, I really don't. I'd much rather build with Stable or Long Term Support for a lot of things, and dip into the riskier end of the pool for specific things if I need to.)
>Though I have wondered about the idea of programming something like Dependabot, but telling it, hey, tell me about known CVEs and security releases, but otherwise, let things cook for 6 months before automatically building a PR for me to update.
I have only skimmed the docs and I wouldn't be completely shocked if there's a "wait X weeks/months to notify about non-security updates" but I don't know about it if it's there. If it is there, hey, great! Won't be the first time I really wished that X did Y and found out that yes it already does.
Good questions. I suppose both of these issues could solved using standard x.509 certificates, which may be the next step for TimestampIt. I wanted to avoid using certificates to keep things more simple but it does leave open some issues.
To more directly answer your questions:
> 1. What would happen in the event when the private key was leaked? Is there any scenario for invalidating or rotating keys?
Right now there is no good means of invalidating a key. There is key rotation however, and upon key rotation the private key is completely deleted from all places it is stored. So I am doing everything I can to minimize exposure of the private key, but of course nothing is fool proof.
> 2. For verifying the proof file, it seems like the URL for obtaining the signature key[1] is read directly off the proof file[2]. Wouldn’t that allow an attacker to publish proof files that point to their own server, hence allowing the attacker to sign fabricated files and still make them pass the automated validation?
This is a good call out. The verification scripts should check that the key-url has a trusted domain on it. I will make those changes soon.
One of Rails most under appreciated features is testing. Basically everything you can do in Rails has a way to write an automated test for it. Having good tests is a way to keep momentum as the project grows in complexity.
I love Rails. Every investment I've made in learning it has yielded returns for years. Everytime I learn a new JS framework, its obsolete two years later.
That sounds really interesting. I'm working on a similar autotesting module for my Django project. Would you mind describing essentially how it works and it's capabilities? There are a lot of great ideas in Rails, but I've never had an excuse to learn it.
Also, is automated version bumps really such a good thing? Many times I have wasted hours tracking down a bug that was introduced by bumping library. Sometimes only the patch version of the library is different so it shouldn't be breaking anything... but it does! It is so much better to update intentionally, test, deploy. Though this does assume you have a modest number of dependencies which pretty much excludes any kind of server-side javascript project.