The Deep North: August 2024

Thursday, August 29, 2024

Persistent Misunderstandings in Software Development

Yikes. This was an answer to a Quora question about development myths and I just kept hammering them out because I have seen a lot of critical misunderstandings in my decades of programming experience. I could just keep going and going. There's a book to be had in explaining all the many misapprehensions of journeyman developers, why they are incorrect and what solutions (if any) apply in dealing with them and their results.

Persistent Misunderstandings in Software Development:

Things Won’t Change: The mistaken belief that the initial project requirements, timeline, and scope will remain constant throughout the development process.
Nothing Will Go Wrong: The expectation that the development process will proceed smoothly without unforeseen challenges, bugs, or setbacks.
Timeline Predictions Are Reliable: The assumption that you can accurately predict timelines and outcomes for problems that are yet to be fully understood or defined.
Human Factors Don’t Matter: Ignoring the reality that developers are human beings with emotions, external responsibilities, and varying productivity levels.
Developers Are Interchangeable: The belief that any developer can be easily replaced with another without impacting the project's progress or quality.
Testing All Pathways Isn’t Necessary: The dangerous assumption that certain software pathways don’t need to be tested because they are unlikely to be encountered.
Rare Issues Won’t Happen: The flawed logic that if something is unlikely, it can be safely ignored.
Multiple Entrances/Exits Are Acceptable: The idea that code can have multiple points of entry and exit without introducing complexity and errors.
Uncontrolled Aborts Are Preferable: The misconception that sudden, uncontrolled aborts are better than controlled unwinding with appropriate logging or recovery mechanisms.
Logging Can Be Skipped: The belief that comprehensive logging isn’t necessary for non-trivial production software.
Premature Optimization Is Safe: The persistent misunderstanding that optimizing early in the development process is beneficial without considering the impact on future changes.
Failing to Optimize Isn’t Harmful: Conversely, the belief that neglecting necessary optimization won’t have significant negative consequences.
More Developers = Faster Delivery: The fallacy that adding more developers will proportionally speed up project completion, akin to thinking nine women can produce a baby in one month.
You Can Fully Understand Requirements Upfront: The expectation that all requirements can be perfectly understood and specified before development begins.
You Can Design Perfectly Before Coding: The belief that it’s possible to design a flawless system architecture before any coding starts.
Regression Testing Can Be Omitted: The mistaken belief that full regression testing isn’t necessary for ensuring software stability.
Delivery Systems Are Homogeneous: The assumption that all delivery systems will behave consistently, ignoring potential variability and edge cases.
Function and Budget Can Be Set Beforehand: The expectation that both the delivered functionality and budget can be fixed before significant development work begins.
Developers and Users Always Understand Each Other: The belief that developers and users are always on the same page without the need for tangible, usable software to bridge understanding.
Floating-Point Arithmetic Is Reliable: The misunderstanding that floating-point arithmetic will always yield consistent results without the need for careful handling and testing.
Rounding Is Consistent Everywhere: The erroneous assumption that rounding operations are consistent across all platforms and software environments.
Human Language Is Precise Enough for Code: The belief that human language is sufficient for specifying code without ambiguity or misinterpretation.
Precision Isn’t Necessary: The notion that you can develop software without rigorous precision, understanding, and thorough testing.
It’s Always Feasible: The overconfidence that every project is doable without significant risks or obstacles.
Security Isn’t a Priority: The dangerous belief that security concerns can be overlooked, or that some attack vectors aren’t worth addressing.
Nobody Will Let You Down: The unrealistic expectation that no team member will face personal issues, illness, or other setbacks during the project.
Your Project Will Survive: The assumption that your project is immune to cancellation or major changes before completion.
Future Tech Predictions Are Accurate: The belief that you can accurately predict the future state of technology and its impact on your project.
Newer Is Better: The naive belief that the latest technology is automatically superior and should be used without question.
Success Is Guaranteed If It Works Once: The damaging notion that finding one way the software behaves correctly is enough, rather than ensuring all potential failure points are addressed. This includes the irritating response, "It works on my machine," which shifts the blame to users instead of addressing the fragility of the software.
Unit Tests Are Enough: The mistaken belief that unit, integration, and system tests can fully substitute for real-world testing with actual users, in pilot phases, and during rollout.
Tool Output Equals Correctness: The belief that if development tools don’t flag issues, the software is automatically correct, ignoring the need for deeper verification.
Unpredicted Issues Won’t Arise: The dangerous oversight that entirely unpredicted and intrinsically unpredictable issues won’t emerge.
Projects Always Finish on Time: The optimistic belief that projects will meet deadlines, despite the well-known tendency for timelines to slip.
Overconfidence in Estimations: The frequent error of underestimating the time and effort required, leading to projects dragging on much longer than anticipated.
You Always Know What You’re Doing: The hubris of believing that you fully understand the problem and that confidence alone will lead to success, without acknowledging the complexities involved.
Resources Won’t Run Out: The assumption that time, budget, or energy won’t run out before the project is complete.
Documentation Will Match the System: The unrealistic belief that documentation will be perfectly in sync with the system at the time of delivery.
People Will Notice What’s Done Right: The expectation that users and stakeholders will recognize what has been done well, rather than focusing solely on deficiencies.
Premature Release Won’t Happen: The common situation where management forces an unfinished or hacked-together solution into production.
Management Will Understand: The assumption that management or stakeholders will fully understand the technical reasons why the software isn’t ready.
Murphy’s Law Is Just an Adage: Misunderstanding Murphy’s Law as a mere saying rather than acknowledging it as a genuine mathematical reality that affects software development.
Dependencies Will Just Work: Underestimating the challenges posed by software dependencies, assuming that everything will work together seamlessly without conflicts.
Libraries Will Solve Everything: The belief that third-party libraries or frameworks will solve all problems without introducing new ones or creating additional complexity.
Scalability Will Handle Itself: The assumption that software designed for small-scale use will automatically scale to handle larger loads without significant rework.
Documentation Can Wait: The belief that documentation can be written after the code is complete without compromising its accuracy or usefulness.
Single Points of Failure Are Fine: Ignoring the risks associated with having single points of failure in the system, assuming they won’t be an issue until they become one.

Why we need to love someone

You are a product of Evolution by Natural Selection. The one thing that got you here was generations before you promoting their design forward through time.

For your design to make it forward through time, you have to ensure the survivability of that design. It has to be reproduced somehow, either by doing something that favors the design in related individuals or by replicating your own design into the next generation.

Our design is primarily transmitted through genetic material and that genetic material makes it into the next generation through the production of offspring by sexual reproduction.

Failing to promote your design forward is fatal. A design that does not survive is a bad design that … does not survive. You are the product of many, many, *many* generations of good design.

How something like romantic love and yearning for it arises can get extremely complex, but it should be apparent that males and females desiring contact will get together. And that, at least in many cases, eventually results in offspring. The design moves forward once again in time.

Seeking a mate can indeed be very challenging, ‘an arduous process’. In fact, even for those with the most resources, it is still a challenge because you are looking for the optimal strategy and that can be extremely difficult to determine. Some of us will have no problem getting some kind of offspring into the next generation, but this is a competition and just getting to the next generation is necessary but not sufficient. You need to produce *viable* offspring that will survive, reproduce and keep their genes alive.

You are stuck with the longing, I am afraid. Make the best of it. Don’t drift about waiting for someone to come your way or waste tons of time in trial and error. Do your research. Learn how to be as attractive a mate as you can be. Learn how to find potential mates and identify which ones are reasonable choices.

Potential mates are looking for ‘fitness’. Mates are seeking ‘attractive’ individuals. What makes people attractive are signs of fitness for the purpose of conceiving and raising successful children. For males, ‘attractive’ means females who are healthy and have maximal childbearing potential. What makes people appear ‘beautiful’ is in fact just overt signs of viability and physical health. Signs of this are regular features, appropriate dimensions, vigor, good teeth, healthy hair, etc. For males, the optimal female is one on the low end of childbearing age. Last I heard in populations near me that means women around 20 to 23. For females, ‘attractive’ means males that are physically attractive as above (health, regular features, etc). However, they favor males that demonstrate the means and desire to stay with them to protect and support a family. Last I heard, men are at their most attractive to women when they are in the range of 29–39.

For both men and women things like high social status and wealth are attractive.

Empirically, animals tend to favor mates that are similar in appearance to themselves. This makes sense because deviation from norms is a warning sign more often than not. There is an effect where an individual that stands out is *more* attractive to some, but the smart money is on fitting in.

It is best if you *are* the picture of health, wonderfully good looking, come from a great family, are wealthy and well-connected. This is why princes and princesses figure prominently in romantic narratives. Given a choice, most people are going to opt for the beautiful, wealthy and powerful at the pinnacle of nobility.

Most of us are not nearly ideal. However, the appearance of the ideal is a really good substitute. Women can’t always *be* 21 years old, but by plumping and reddening their lips, applying makeup to smooth their skin, wearing cleverly arranged clothing, adopting certain mannerisms, etc, they can appear closer to that ideal. Men can’t always be 30 years old, but by careful grooming, staying in shape and similarly wearing clever clothing they can appear closer to the ideal.

For men, it helps to have that expensive car and fancy suit that demonstrate the means to provide for a family. Being able to escort women to important engagements, fancy restaurants, exclusive parties, etc indicate a desirable social status.

There are all sorts of tips and tricks to enhance your appearance to a potential mate. For women, something as simple as pinching their cheeks before heading out the door can give them a tiny edge.

Women should be mindful of the narrow window they have to be at their most attractive and should take pains to make sure they ‘play the field’ early enough to get an idea of what is out there before they hit their peak years.

Men are similarly affected by a window, but it is a larger window (though not all that much) and it appears later. If you are a man, you have more time, but a word to the wise: you will need it.

To maximize the process of searching for and acquiring a mate, you need to move quickly to establish what is out there and how you stack up to the competition. You have to correct any deficiencies you can, settle into a search for suitable candidates and then make their acquaintance. Once you are there, you are looking for someone that you can actually stay with. They may be gorgeous to look at, but still not right for you. You have to put your best foot forward, but you also have to be reasonably honest and authentic.

Chemistry is important. I can’t prove it, but I think the reason that being ‘a good kisser’ is a big deal is because it is a vetting process — literally a chemical analysis — to determine that there is a good genetic match there.

You want to create ‘limerence’ in the right candidate individual and you want to experience limerence yourself. If you have been careful you may well find that it springs up on you without any effort. Otherwise, you need to use everything you’ve got to create it. Mutual infatuation can be extremely annoying to the people around you, but for the couple in love, it is pretty awesome and well worth some effort.

You need, as I say, to put your best foot forward, but that does not mean outrageous cheating. The ‘PUA’ movement is the antithesis of romance. The whole premise of that movement is to prey on women by tricking them into sexual involvement under false pretenses. It is a sad strategy, for sad men that only spreads misery. Don’t be that guy.

Instagram Account Banned

This was resolved as simply a mistake on the part of Instagram, but the hint below might help someone.

Can't get a texted code number from Instagram? You might need to unblock one of their numbers on your phone. In my case, unblocking 326-65 allowed me to get the code I needed to continue with their horribly hostile broken system.

I'm 'a computer guy'. I have been on social systems in cyberspace since before the worldwide web. Since, even, before the term 'cyberspace' came and went in popular geek parlance. When younger, long ago, I could be a bit sharp and sarcastic online, but I never intended to be mean. Before the WWW came along I was already kinder and gentler. I registered my first domain name in 1987, and for a while did websites for people starting in 1999. I have been a citizen in cyberspace longer than most of the people who have seized control of it have been alive. I have been a kind, mannerly, and 'community-guidelines' friendly citizen of the web since its very beginning. I don't transgress in online communities and any community managers that believe otherwise are mistaken. It's on them. If there is something objectionable associated with my account online it is their security breaking down, not my manners.

I was invited to review an Instagram post on the account of an important public person. By co-incidence, my account, which I hardly ever access was banned by Instagram. I could not log on and was given the option to enter an appeal. However, the system demanded that I put in a code that it would send by SMS text to my phone for me to even register my protest. I tried a number of times unsuccessfully to get the code sent, but it never arrived. After a few attempts it said I had used up my chances for SMS and had to wait for 24 hours to try again. I waited. I tried again, still without luck.

The system offered no other way to protest the ban or to even contact Instagram. I tried to get them on Twitter/X without luck. I logged in to Meta to try, tried to use Facebook to log in, all to no avail.

My posting on X prompted replies by accounts clearly charging people money to fix this issue. Having my time wasted and paying for the privilege was 'a bridge too far' for me.

Instagram's system is ridiculously hostile to just about anyone, it would seem. For ordinary people who have invested their time in their account it must be a nightmare.

I went looking for a solution to the defective SMS situation and found something that seemed an improbable fix. Apparently it is impossible to get the SMS or any indication of what went wrong if their number is blocked. It is also possible that, for whatever reason, you have blocked the number. I have no idea what happened for me to block that number, but it was blocked on my phone.

The number in question is 326-65. Unblocking that number allowed me to receive the code so that my appeal could continue.

Because I have been something of an insider in many places, am a technical person myself, and have friends and associates that are technical, I have been able to deal with various frustrations over the years. However, I note with alarm that the stranglehold large incumbents have on the world's networks is beginning to make me, one of the people who designed and built parts of the infrastructure, just another supplicant at the mercy of a merciless oligarchic bureaucracy.

I believe that there is, in the medium term of about five years, a way that the citizens of the world can free themselves from what is rapidly becoming control by an increasingly hostile and tyrannical minority. I am working on aspects of this. For now, though, I would say to people to get off the sidelines, get involved, and remember your neighbor is not the enemy.

Truth, Belief, and Notions about Reality

"It Ain’t What You Don’t Know That Gets You Into Trouble. It’s What You Know for Sure That Just Ain’t So" (variously attributed).

A Thoughtful Reflection on Belief and Understanding

Have you ever found yourself mistaken about something? If so, you have a firsthand example of why belief may not be the most reliable guide. Personally, I choose to believe in nothing, as I see belief itself as an irrational conviction that something is undeniably true, despite the fact that we can never be fully aligned with empirical reality—if such a reality even exists.

Instead, I prefer to rely on models grounded in solid evidence and reason. These models serve as provisional lenses, allowing me to view the world with the best understanding I can muster at any given time. As new data and insights emerge, I’m always ready to adapt and shift my perspective to a better model. When you truly understand something, there’s no need to cling to belief; you can extract value from your model without the need for unwavering faith.

When a superior model presents itself, it’s easy to embrace it, since neither it nor your previous model holds an absolute claim to correctness. The truth is, we simply do not possess complete knowledge, and perhaps achieving that level of understanding is inherently impossible.

Now, consider the concept of unicorns or even the idea of God. They may not manifest in ways that meet our expectations, but they certainly exist in the realm of language and thought. It’s not merely about existence; it’s about how these concepts are expressed and perceived. While I don’t anticipate encountering a unicorn in real life, I wouldn’t let that impede my understanding. At this point in my journey, it’s just another fascinating data point.

Ultimately, we shape the world we observe within our minds. What you perceive is merely a model of reality, not reality itself. Some argue that we inhabit a universe with dimensions beyond the three we typically acknowledge, suggesting that we’re only seeing a faint, incomplete shadow of the complex world around us.

Democracy is Problematic

Derek Muller (Veritasium) made an interesting video about issues with democracy and voting systems. It is an important discussion which I hope he will eventually turn his big brain to more fully articulating issues and possible solutions. As things stand, our political system does not represent the reasonable wishes of the population and even less so their best interests. That has been getting worse at a time where it is ever more important that we get it right.

I have designed a fair, private, but transparent and auditable, voting protocol

Below is the comment I made on his video and below that a link to the video which, although somewhat disappointing for me I still think is worthwhile to watch.

You are a smart guy. Fix this. There are things that have not been accounted for here such as proxies and a favorite of mine to push decisions closer to the 'ground' of the individual voter. There is no conflict between me and another person as to what to have for dinner because I decide myself for myself. Some decisions need to be made thoughtfully by people who both care and understand. That is not most people, nor should it be. We should not all be voting on the test for how much tension a screw in a Rocketship should bear. I'm a smart guy, but I don't even know if that question is even a sensible one to ask. Most importantly at this stage of the game is that we globally exist within an ancient power regime that takes as foundational premise that some are better than and more entitled than others and that the allocation of the world's resources is properly to those who have a legal claim to the power of the state to enforce ownership and privilege. We have been sold the notion that we must all somehow 'earn' our right to breath the air and that it is okay that people unable to do so suffocate. It's messed up when I use the example 'air', but since we apply that to food and shelter and the variety of means to have a life, it's the same thing. You won't die as quickly if you starve to death, but you will die, and in a world of abundance such as we have there is no need for that except to uphold the principle that there are people who 'own' more than they need and people who do not own enough to meet their needs. I designed a voting system for a municipal election in Ontario whereby people's votes could be secret, known only to them, yet at the same time be available so the voter could ensure that their particular vote was counted as cast and the published results would allow others to do the same and it could be verified that only legitimate voters voted, and in the event of any irregularity or dispute the entire election could be audited to ensure total compliance still without revealing how a vote was associated with an individual. We had a date and time set to present this to the city council. The presentation was cancelled because someone on the council had a relative at another company that ultimately was given the go-ahead without ever seeing what we proposed. Subsequently that supplier was accused of irregularities (just errors, I think), go figure. We now have, in first world countries like Canada, the facilities to hold elections with such a mechanism and if things like ranked-choice or some better variant are chosen then the same thing would apply such that people could verify that their vote was counted as cast and even with a relatively complex voting protocol the system overall could be vetted after the fact. One of the things not addressed here is a foundational constitution that provides for inviolable individual rights and entitlements that even a majority cannot vote away. It does not address genuine equality of opportunity. If someone is simply not educated enough to render a sensible opinion, or if someone is struggling to keep body and soul together where an election is the least of their worries or if methods of voting are sufficiently onerous as to discourage people, or if candidates have such extremely disparate resources that only one is ever presented to people ... well ... you get the idea ... a real system requires thought beyond what has been presented as a mathematical problem. We need a rational constitution and a real voting system because we are entering a time where the entire worlds jurisdictions and all of their rules are about to be turned on their heads as automation eliminates work, we enter a new time of abundance, and our impact on our environment becomes ever more extreme.

Link: https://www.youtube.com/watch?v=qf7ws2DF-zk

Wednesday, August 14, 2024

Apparent Annual Cost of Climate Action

It seems that no matter how far out the Climate Crisis Narrative gets, it persists with an almost unshakable momentum. The recent statement by The Climate Policy Initiative calls for all the money in the world—and more—to be spent on climate change action. It’s a staggering proposal that, upon closer inspection, raises serious questions.

A closer look suggests they might be advocating for annual expenditures five times higher than current levels, totaling around $5 trillion. While this might be achievable, is it reasonable? That money could instead be directed toward critical needs like healthcare and ensuring that children don’t go to bed hungry. How did we reach a point where prioritizing global climate initiatives over immediate humanitarian needs seems justifiable?

I’ve spent time delving into this issue, despite its apparent contradictions. The reality is that nothing happening now is truly "unprecedented." Life on Earth has persisted for billions of years, adapting to a constantly changing climate, including fluctuations in global temperature, ice cover, and CO2 concentrations. Change is a constant, and the living world is more than capable of adapting.

Since the Little Ice Age, the world has experienced gentle warming—an entirely natural and non-catastrophic process. Over the past century, we’ve increased CO2 levels in the atmosphere by liberating carbon from ancient plant matter. This rise in CO2 has led to a greening of the Earth, as plant life flourishes with more available carbon dioxide. This is not an opinion; it’s an observable fact.

There are speculative concerns about CO2-driven warming and its potential downsides, but the evidence for catastrophic outcomes is far from convincing. We’ve dealt with warmer periods and higher sea levels before, and we can do so again if necessary.

Looking forward, I hope that as AI technology improves, it will help us sift through the noise of the past few decades and provide a clearer picture of how the climate is changing—and whether the resources we’re dedicating to this issue are truly warranted.

A good starting point for this inquiry is the null hypothesis for catastrophic global warming:

"There is no statistically significant evidence that the current rate of global warming will lead to catastrophic outcomes that significantly exceed the natural variability and resilience of the Earth's climate system and biosphere, within a timeframe relevant to human society."

By framing the debate in terms of this null hypothesis, we encourage a more rigorous examination of the evidence and promote a balanced approach to understanding the potential risks and uncertainties associated with global climate change.

Source article: https://www.climatepolicyinitiative.org/press-release/annual-finance-for-climate-action-surpasses-usd-1-trillion-but-far-from-levels-needed-to-avoid-devastating-future-losses/

AI might help with this in the future

I have hopes that as it improves AI will help us cut through all the noise of the last few decades to come up with a realistic picture of how the Climate is changing and whether or not, all things considered, we should be spending all this time, effort, and resources on this issue alone. Below is an exchange with ChatGPT 4o:

Me: What do you reckon would be a good statement of the null hypothesis when it comes to catastrophic global warming?

ChatGPT: A good statement of the null hypothesis in the context of catastrophic global warming would be:

Key Points to Consider:

Statistical Significance: The null hypothesis asserts that any observed changes in global climate metrics (e.g., temperature, sea level rise) are within the range of natural variability and not indicative of a catastrophic trend directly attributable to anthropogenic factors.
Natural Variability: This hypothesis assumes that the Earth's climate has always undergone fluctuations due to various natural factors (e.g., solar cycles, volcanic activity), and it suggests that the current warming trend may not be exceptional when viewed in a longer historical context.
Resilience of the Earth's Systems: The null hypothesis considers that Earth's climate and ecosystems have inherent mechanisms of resilience and adaptation, which could mitigate the potential impacts of warming without leading to catastrophic consequences.
Relevance to Human Society: The null hypothesis implies that even if warming continues, it may not result in outcomes that are catastrophic within the timeframe that is most relevant to human societies, such as the next few centuries.

Testing the Null Hypothesis:

To reject this null hypothesis, proponents of the catastrophic global warming narrative would need to provide strong evidence showing that:

The current warming trend is unprecedented and outside the bounds of natural variability.
The impacts of this warming are demonstrably leading to irreversible and severe damage to global ecosystems and human societies.
The resilience and adaptive capacities of the Earth's systems are overwhelmed, leading to outcomes that can be classified as catastrophic.

By framing the debate in terms of this null hypothesis, it encourages a more rigorous examination of the evidence and promotes a balanced approach to understanding the potential risks and uncertainties associated with global climate change.