Note: I believe I did a good job in describing my journey to dealing with this issue in the PyMarkdown_Test project, so I decided to cheat a little and just include it here. Hope any readers do not mind.
It may not be an elephant to other people, it is to me. For me, the elephant that I have not been talking about is a poor release process for the PyMarkdown project.
The thing that has been bothering me for the last three to four weeks are the errors I made in releasing version 0.9.10 of the PyMarkdown project. I dropped the ball in effectively testing the product before its release, and I feel bad about it. It is even more embarrassing that I had something in place, and it failed to recognize any errors.
Ever since the first release of the PyMarkdown project, I have had a local directory on my machine called “pym_test”. In that directory, I had a couple of batch scripts that reach over to the PyMarkdown project, install the release packages into the local directory, and execute tests of PyMarkdown. These tests were simple tests, but they made sure that the releases of PyMarkdown went smoothly.
Until they didn’t.
Sometime after release 0.9.5, there were a couple of times where I caught something just before or after it was released and was able to mitigate it. And then the release of 0.9.10 happened. For whatever reason (still researching), that release worked fine on my machine, but failed on other machines.
My test scripts were mainly testing a packaged PyMarkdown, with only a passing glance at the Pre-Commit usage of PyMarkdown as a pre-commit hook. To accommodate a required increase in one of the project’s dependent packages, I changed the
Piplockfile to access the changed API and tested locally with that configuration. All the scenario tests passed, so I thought all was good. Even after running my local test scripts, all tests were passing and everything looked good.After doing some debugging, I found out that I had changed the
Piplockfile, but I had not changed theinstall-requirements.txtfile used by the package install scripts. This resulted in both package installations and the Pre-Commit installations failing. My cobbled together tests on my machine were passing. Installed on another machine, those same tests were failing. There was a problem.
That is how the PyMarkdown_Test project was started. The project goes into more details on how I solved those problems, but I am feeling a lot better about the status of the release tests. And since the PyMarkdown workflow kicks off the PyMarkdown_Test workflow under certain conditions, I know that it is going to be executed with enough frequency to hopefully avoid situations like release 0.9.10.
I might play around… er… experiment scientifically on fine tuning bits of the new project, but right now I am happy with how it is performing. But I am hoping to make progress on other projects that I am working on, as well as getting some more of the PyMarkdown scenario tests passing. Stay Tuned!
Comments
So what do you think? Did I miss something? Is any part unclear? Leave your comments below.