Better by our own standards

This book review in the New Yorker of Charles King’s “Gods of the Upper Air” has been kicking about my brain for a long while, especially in regards to the phrase “variations within groups are greater than variations between groups” concretely in my work on accessibility, diversity, equity and inclusion.

The evidence proved, Boas said, “the plasticity of human types.” It also showed that variations within groups are greater than variations between groups.

Boas devoted his life to showing people that the science they were relying on was bad science. “He believed the world must be made safe for differences,” Ruth Benedict wrote when Boas died.

It’s true that Boas and Benedict spoke of “relativity,” and that at the end of “Patterns of Culture” Benedict refers to “coexisting and equally valid patterns of life which mankind has created for itself from the raw materials of existence.” But everything else in Benedict’s book contradicts the assertion that all cultures are “equally valid.” The whole point is to judge which practices, others’ or our own, seem to produce the kind of society we want. The anthropological mirror has a moral purpose.

The term “culture” is responsible for some of the confusion. We think that to call something part of a group’s culture is to excuse it from judgment. We say, That’s just the lens through which people in that society view the world. It’s not for us to tell them what to think. Our ways are not better, only different. What it all boils down to (to paraphrase Montaigne) is: We wear pants; they do not. That would be relativism.

But to say that a belief or a practice is culture-relative is not to place it beyond judgment. The whole force of Boasian anthropology is the demonstration that racial prejudice is cultural. The belief that some races are superior and some inferior is learned; it has no basis in biology. It is therefore subject to criticism.

Boas spent his entire life telling people that intolerance is wrong. King says that cultural anthropology pushes us to expand our notion of the human. That may be so, but it has nothing to do with relativism. King’s anthropologists are prescriptivists. They are constantly telling us to unlearn one way of living in order to learn a way that is better by our own standards.


The five ideals explained

I previously posted the The Three Ways from The Phoenix Project. Here are the Five Ideals from The Unicorn Project, by way of Gene Kim’s IT Revolution blog:

The First Ideal: Locality and Simplicity

We need to design things so that we have locality in our systems and the organizations that build them. We need simplicity in everything we do. This ideal relates to the degree to which a development team can make local code changes in a single location without impacting various teams. The last place we want complexity is internally, whether it’s in our code, in our organization, or in our processes.

The Second Ideal: Focus, Flow, and Joy

The Second Ideal is all about how our daily work feels. Is our work marked by boredom and waiting for other people to get things done on our behalf? Do we blindly work on small pieces of the whole, only seeing the outcomes of our work during deployment when everything blows up, leading to firefighting, punishment, and burnout? Or do we work in small batches, ideally single-piece flow, getting fast and continual feedback on our work? These are the conditions that allow for focus and flow, challenge, learning, discovery, mastering our domain, and even joy. This is what being a developer means.

The Third Ideal: Improvement of Daily Work

The Third Ideal addresses paying down technical debt and improving architecture. When technical debt is treated as a priority and paid down and architecture is continuously improved and modernized, teams can work with flow, delivering better value sooner, safer, and happier. And the business ultimately wins when developers can deliver on enterprise performance goals.

The Fourth Ideal: Psychological Safety

Psychological safety is one of the top predictors of team performance. When team members feel safe to talk about problems, problems can not only be fixed but prevented. Solving problems requires honesty, and honesty requires an absence of fear. In knowledge work, psychological safety should be treated with the same importance as physical safety is in manufacturing.

The Fifth Ideal: Customer Focus

Customer focus relates to the difference between core and context as defined by Geoffrey Moore. Core is what customers are willing and able to pay for, the bread and butter of your business. Context is what they don’t care about, what it took to get them that product, including all the backend systems of an organization like HR and marketing and development. It’s critical to look at these Context systems as essential, as mission critical, and fund them appropriately. Context should never kill core.

And a shorter version that’s less engineer focused from “DevOps: A Primer For The Business Leader”:

  • Locality and Simplicity: alleviate dependencies between teams and components.
  • Focus, Flow, and Joy: the smooth flow of work that enables focus and joy.
  • Improvement of Daily Work: continuously improve and pay down technical debt.
  • Psychological Safety: a top predictor of team performance; enables improvement.
  • Customer Focus: optimize for customer value, not for a role-based silo.

The furthest to go in efforts to reach their full humanity

Tawana “Honeycomb” Petty writing in a “Mama Lila Cabbil Tribute: Pushing Us Toward Deeper Thought, Still”, published in Riverwise Magazine.

Through my organizing and teaching, I have asked that anti-racist circles step away from performative testimonials of privilege for whites and lack of privilege for Black and Brown people. I have asked that all allies move from ally-ship towards co-liberation, with the belief that we can only make systemic change if we understand our liberation is tied up with one another’s. My historical and current analysis of this moment pushes me to interrogate the notion that (most) white people “benefit” from their forced relationship with white supremacy.

I have asked that white allies (aspiring co-liberators) seek to understand the impact that the myth of white superiority and the system of white supremacy has had on their own communities. I have asked them to deal with school shootings in their communities, opioid abuse, domestic violence, and rising incidents of suicide. I have asked them if they truly believe what they say when they testify to their privilege.

For many, this way of thinking may not appear to answer the questions that plague Black and Brown people in America. However, I am of the mindset that a dehumanized being cannot see another as fully human. I am of the mindset that the white children who are shooting up schools have fallen victim to trying to live up to the myth of white superiority.

White men, even in progressive circles, are taught they are superior to white women, Black women, and every other living being on the planet. What would it mean for the anti-racism movement to teach white men that they have the furthest to go in efforts to reach their full humanity? What would it look like if they don’t enter spaces acting inherently superior (privileged), but rather with a lot of work to do to shed the legacy of violence that comes with their perception of superiority?


The sublime

From Susan Casey’s The wave:

Teahupoo, with its timeless power, brought to mind the age-old philosophical quest to distinguish between beauty and its twisted cousin, the sublime: for the merely pretty to graduate to the sublime, terror was required in the mix. “The Alps fill the mind with a kind of agreeable horror,” wrote one seventeenth-century thinker, summing up the concept. And while humans were capable of creating the lovely, the dramatic, the sad, or the inspiring, only nature could produce the sublime. It was a concept both comforting and disturbing: there are many things out there more powerful than we are. No one was more aware of this than the men who’d ridden Teahupoo on this day (except, perhaps, the ones who had fallen on it).


Centering Civic Tech

From Cyd Harrell’s excellent “A Civic Technoligists Practice Guide”, via a Twitter convo, reformatted by me:

Because its goal is change, civic tech embodies an interesting split between demonstrating and operationalizing the potential of modern tech. I like to call these two branches showing what’s possible and doing what’s necessary. Many projects are a mix of the two, but they require different mindsets.

“Showing what’s possible” is about speed, prototyping, design, public feedback, and data. These are often web projects because web tools are great for those purposes.

“Doing what’s necessary,” on the other hand, is about shifting the underlying practices and systems: back-end systems, security, and procurement; hiring and team composition; even shifting budget priorities.”

[…]

But our job as civic technologists isn’t to be the hero of the stories we stumble into halfway through; it’s to understand and support the people who have already been in place doing the work, and who want to use tech to make improvements.

They line up with Code for America’s pillars of “Show what’s possible”, “Help people do it themselves” and “Build a movement” (though the latter is rather more grandiose than understand and support).

As we called these in my previous career, Direct Service, Capacity Building, and Roy Johnson (“Put some gratitude in your attitude!”).


GoodJob v1.3: Web dashboard and full documentation

GoodJob version 1.3 is released. GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails. If you’re new to GoodJob, read the introductory blog post.

GoodJob’s v1.3 release adds a mountable Web Dashboard and improved README documentation and complete code-level YARD documentation.

Version 1.3’s release comes five weeks after v1.2 and three months after GoodJob’s initial v1.0 release.

Shoutouts 🙌

GoodJob has accepted contributions from 9 people total, and currently has 559 stars on Github. The project just passed 150 combined Issues and Pull Requests.

I’m grateful for everyone who has reached out to me on Ruby on Rails Link Slack (@bensheldon) and Twitter (@bensheldon)

🙏

Mountable web dashboard

GoodJob v1.3 adds a web dashboard for exploring and visualizing job status and queue health.

GoodJob Dashboard MVP

The web dashboard is implemented as an optional Rails::Engine and includes charts and lists of pending jobs.

I expect the web dashboard to be a hot area of ongoing improvement. This initial release contains a minimum functional interface, a chart, and some necessities like keyset pagination. The dashboard is familiar to develop (Rails Controllers, ERB Views and ActiveRecord), and I’ve adopted Bootstrap CSS and Chartist to ease and speed development.

Improved documentation

GoodJob’s README has been edited and rewritten for clarity and comprehensiveness, and GoodJob’s implementation code is now thoroughly documented with YARD.

Good documentation is vital for open source projects like GoodJob. I worked with Rob Brackett, who consults on complex software and open source challenges.

Upcoming

In the next release, v1.4, I plan to continue adding views and charts to the web dashboard and improving thread management.

Contribute

Code, documentation, and curiosity-based contributions are welcome! Check out the GoodJob Backlog, comment on or open a Github Issue, or make a Pull Request.

I also have a GitHub Sponsors Profile if you’re able to support GoodJob and me monetarily. It helps me stay in touch and send you project updates too.


Performance facilitators not supervisors

Doublespeak from Pro-Publica’s “Meet the Customer Service Reps for Disney and Airbnb Who Have to Pay to Talk to You”, via Pluralistic’s ongoing chickenization coverage:

Arise carefully monitors the language agents use to reinforce that it does not have an employment relationship with them. Stung by lawsuits that claimed Arise had actually employed agents but didn’t pay them fairly, Arise’s legal department has become a kind of word police, one former staffer told ProPublica.

“You don’t schedule ‘hours,’ you schedule ‘intervals,’” the former staffer said. Agents were not to be addressed as “you,” but “your business.” They were not “working,” they were “servicing.” There were no “supervisors,” only “performance facilitators.” Agents were not “coached.” Rather, their services were “enhanced.”

Once, an Arise manager, testifying in an arbitration hearing, was asked about meetings that performance facilitators have with agents. “They’re not meetings,” he said. “They’re informational sessions, or hosts.”

In an internal announcement in 2012, Arise listed “new terminology” for eight terms to avoid “the misconception” that agents are Arise employees. The corporate link between Arise and the agents went from being called Virtual Services Corporations to Independent Businesses. Service Fees became Service Revenue. Central Operations became Support Operations.

Arise seems particularly unable to settle on a term for the agents. Early on, the company called each a CyberAgent. Later came Arise Certified Professional. In 2012, that was changed to Client Support Professional. Nowadays, Arise’s website calls agents “onshore brand advocates or Service Partners.”

“Arise-speak,” as one opposing attorney called it in legal proceedings, could be a wonder to behold. Client Support Professionals (CSPs) would work with Quality Assurance Performance Facilitators (QAPFs) in a Performance Enhancement Session (PES), or they might reach out to Chat Performance Facilitators or Escalation Performance Facilitators, and none would be an Arise employee, all would be independent contractors.

And a disturbing exchange:

Rice said he worked out of his bedroom, in his mother’s home, helping customers for Arise’s clients, including Barnes & Noble, Disney and Sears. While testifying, Rice referred to performance facilitators in the Arise network as supervisors. This elicited a challenge from a lawyer for Arise.

“Where’d you get that term from, supervisor?” the lawyer asked.

“Growing up in America,” Rice said. “That’s the term people use for people that are above you.”

“… You never referred to them as supervisors while you were providing services, did you?”

“Well, yeah,” Rice said.

“You did? To who?”

“Well, obviously I’m on the phone with a customer, I’m not going to say, ‘OK, let me go check my chat performance facilitator.’ Usually I just said, you know, ‘Let me just talk to my supervisor.’”


Respect our vendors

Costco Values

“Respect our vendors” is foreign enough to me in software engineering that I took this picture of Costco’s values. The opposite of respect, perhaps “contempt for vendors and tools,” seems endemic.

At one job, memorably, a coworker was fired over it. Our engineering team had a shared email list used when setting up root accounts on various 3rd party services, including our primary infrastructure vendor for whom we were one of their largest customers.

The infrastructure vendor sent a Net-Promoter Score-like survey to our shared email list. Receiving this kind of marketing junk was frequent enough that I ignored it, but my colleague filled it out:

On a scale of 1-10, how likely are you to recommend our service?: “1”

Why?: “I hate you.”

This survey response led to the vendor’s account executive making a frantic and fearful call to our leadership team. The blowback of that led to my coworkers’ termination. (This was not my colleague’s first warning; my team and I were also targets of their trolling and bullying.)

This is a funny story to reflect on. And it’s terribly toxic the multitude of things engineers will despise, wear on their sleeve, and eagerly share at the slightest opportunity, myself included.


GoodJob v1.2: Multithreaded queue isolation and LISTEN/NOTIFY

GoodJob version 1.2 has been released. GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails. If you’re new to GoodJob, read the introductory blog post.

GoodJob’s v1.2 release adds multithreaded queue isolation for easier congestion management, and usage of Postgres LISTEN/NOTIFY to greatly reduce queue latency.

Version 1.2 comes out 2 weeks after GoodJob v1.1, and 5 weeks after GoodJob’s initial v1.0 release.

Multithreaded queue isolation

GoodJob v1.2 adds multithreaded queue isolation for easier congestion management. Queue isolation ensures that slow, long-running jobs do not block the execution of higher priority jobs.

Achieving queue isolation has always been possible by running multiple processes, but GoodJob v1.2 makes it easy to configure multiple isolated thread-pools within a single process.

For example, to create a pool of 2 threads working from the mice queue, and 1 thread working from the elephants queue:

$ bundle exec good_job --queues="mice:2;elephants:1"

Or via an environment variable:

$ GOOD_JOB_QUEUS="mice:2;elephants:1" bundle exec good_job

Additional examples and syntax:

  • --queues=*:2;mice,sparrows:1 will create two thread-pools, one running jobs on any queue, and another dedicated to mice and sparrows queued jobs.
  • --queues=-elephants,whales:2;elephants,whales:1 will create two thread-pools, one running jobs from any queue except the elephants or whales, and another dedicated to elephants and whales queued jobs.

LISTEN/NOTIFY

GoodJob now uses Postgres LISTEN/NOTIFY to push newly enqueued jobs for immediate execution. LISTEN/NOTIFY greatly reduces queue latency, the time between when a job is enqueued and execution begins.

LISTEN/NOTIFY works alongside GoodJob’s polling mechanism. Together, jobs queued for immediate execution (ExampleJob.perform_later) are executed immediately, while future scheduled jobs (ExampleJob.set(wait: 1.hour).perform_later) are executed at (or near) their set time.

Upcoming

In the next release, v1.3, I plan to include a simple web dashboard for inspecting job execution performance, and focus on improving GoodJob’s documentation.

Contribute

Code, documentation, and curiousity-based contributions are welcome! Check out the GoodJob Backlog, comment on or open a Github Issue, or make a Pull Request.

I’ve also set up a GitHub Sponsors Profile if you’re able to support me and GoodJob monetarily. It helps me stay in touch and send you project updates too.


GoodJob v1.1: async and improved documentation

GoodJob version 1.1 has been released. GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails. If you’re new to GoodJob, read the introductory blog post.

GoodJob’s v1.1 release contains a new, economical execution mode called “async” to execute jobs within the webserver process with the same reliability as a separate job worker process.

This release also contains more in-depth documentation based on feedback and questions I’ve received since the v1.0 release.

Version 1.1 comes out 3 weeks after GoodJob v1.0. The initial release of GoodJob was featured on Ruby Weekly, A Fresh Cup

, Awesome Ruby, and was as high as #8 on Hacker News. GoodJob has since received nearly 500 stars on Github.

Async mode

In addition to the $ good_job executable, GoodJob now can execute jobs inside the webserver process itself. For light workloads and simple applications, combining web and worker into a single process is very economical, especially when running on Heroku’s free or hobby plans.

GoodJob’s async execution is compatible with Puma, in multithreaded (RAILS_MAX_THREADS), multi-process (WEB_CONCURRENCY), and memory efficient preload_app! configurations. GoodJob is built with Concurrent Ruby which offers excellent thread and process-forking safety guarantees. Read the GoodJob async documentation for more details.

On a personal level, I’m very excited to have this feature in GoodJob. Async execution was the compelling reason I had previously adopted Que, another Postgres-based backend, in multiple projects and I was heartbroken when Que dropped support for async execution.

Improved documentation

Since GoodJob was released 3 weeks ago, the documentation has been significantly expanded. It contains more code and examples for ensuring reliability and handling job errors. I’ve had dozens of people ask questions through Github Issues and Ruby on Rails Link Slack.

Upcoming

In the next release, v1.2, I plan to simplify the creation of multiple dedicated threadpools within a single process. The goal is to provide an economical solution to congestion when the execution of a number of slow, low-priority jobs (elephants) are being performed and there are no execution resources available for newly introduced fast, high priority jobs (mice) until the currently executing elephants complete.

A proposed configuration, for example:

--queues=mice:2,elephants:4

…would allocate 2 dedicated threads for jobs enqueued on the mice queue, and 4 threads for the elephants queue. Learn more in the feature’s Github Issue.

Contribute

GoodJob continues to be enjoyable to develop and build upon Rails’ ActiveJob and Concurrent Ruby. Contributions are welcomed: check out the GoodJob Backlog, comment on or open a Github Issue, or make a Pull Request.