Shirt sales, scraped daily

I am remiss to finally get around to blogging Day of the Shirt, which I launched back in October, 2010. It’s a straightforward and (hopefully) aesthetically-pleasing t-shirt aggregator.

What’s nifty about Day of the Shirt is that it’s built entirely with  PHP Simple HTML DOM Parser. And when I say entirely, I mean entirely: there is no database backend, everything is scraped, including itself. Day of the Shirt…

  1. scrapes t-shirt vendors to get names, links and thumbnails (which are cropped and cached);

  2. scrapes, parses and rewrites its own DOM when new shirts are added (we’re serving completely static html);

  3. and scrapes itself to compose its daily tweets.

That last step is a little extravagant, but I wanted to separate the early-morning website update from the mid-morning tweeting—otherwise only the early birds would ever see the tweet. Of course, I re-used the tweet-composing algorithm from Panlexicon.

Now the unfortunate part of this project was that I did a fair amount of competition research prior to building this website—but it wasn’t until about a week after I launched that I discovered an equivalent service: Tee MagnetC’est la vie.

Social Media Community Architect and Manager

Exploring the recesses of my email I came across some bad ideas I gave to a good friend, neighbor and excellent “Social Media Community Architect and Manager” as we were exploring possible resume headers for him:

If I were to take the best amalgamation of words, I would go with “Social Media Community Architect and Manager”—which is somewhat awkward. I think you want the words:

  1. social (which is the buzzword of online social networking);

  2. community (which is both online and offline and has a certain fuzziness to it); and

  3. something that describes the process of creation… and management.

I would shy clear of the word technology… “media” definitely has more hotness right now. Maybe “Social Media and Community Architect”.

What about “Social Media and Community Entrepreneur” (everyone loves an Entrepreneur and I would say you qualify more than anyone I know… though it is somewhat heartless)

I went through my contacts on LinkedIn and picked out some words/phrases they use to describe themselves:

  • Community Technologist
  • Online Community Manager
  • strategic planning for your online social network initiatives
  • Building and Executing Social Media Business Strategy
  • Rich Media Developer
  • Interactive Marketing Executive
  • Technology Coordinator
  • Community Architect
  • community & communications coordinator
  • New Media
  • hybrid social media
  • collaboration technology
  • listening technology
  • Emerging
  • Link Development
  • Interactive

I think he went with something sensible like “Community Alchemist”.

Apparently I’m nonprofitly conservative

It’s kind’ve funny how in some contexts I’m a shut-up-and-go radical and in others I’m a hold-your-horses conservative. In response to this article on the Nonprofiteer:

If institutions of higher learning want to maintain their tax-favored status, they should abolish legacy preferences. If they don’t—if they go on practicing white people’s affirmative action—they deserve to be knocked off the comfortable perch on which they now sit.

… I left this comment

I’m sympathetic to the overall sentiment, but I’m always worried when an organization’s activities are judged against an arbitrary measure of “nonprofitness”. The strength of the sector—hard fought over 150 years of legislation and case law—is the breadth of allowable activity. The danger of reform is codifying subjective and contemporary distaste as unlawful behavior, thus limiting the ability of truly transformative organizations to form or function. How might Howard University, for example, find egalitarian social benefit in favoring legacy students?

When an organization’s status as a “nonprofit” is the basis for allegations of hypocrisy I find that the outcome usually furthers the goals of those who wish to do away with the nonprofit sector by doing away with taxes entirely (and by extension their social and redistributive benefits). Which is not to dismiss all nonprofit reform (I am quite in support of transparency and reporting), but the real issue here is not “How can we get more poor kids into Harvard?” but “Harvard should not be your only or best option.”

And I agree with the response I received back from the Nonprofiteer—which goes to the strength of dialogue as tool for navigating between two extremes:

Fair point–and I may have overstated my own (so new?). I certainly am not part of the group you describe which opposes taxes–far from it. But the nonprofit community has long regarded itself as above criticism, and I think the portions of it which foster inequality should be called to account. Perhaps depriving those institutions of their nonprofit status is using a sledgehammer where a scalpel would be preferable–I’m certainly open to that possibility. But that there is an ailment to be addressed strikes me as indisputable.

That national interest thing

I’m still parsing through H.R.1363, the $38 billion appropriations bill passed late last night, but this is generally representative of the sausage trading to pay Paul approach:

Sec. 8079. In addition to the amounts appropriated or otherwise made available elsewhere in this Act, $65,200,000 is hereby appropriated to the Department of Defense: Provided, That upon the determination of the Secretary of Defense that it shall serve the national interest, he shall make grants in the amounts specified as follows: $20,000,000 to the United Service Organizations; $24,000,000 to the Red Cross; $1,200,000 to the Special Olympics; and $20,000,000 to the Youth Mentoring Grants Program: Provided further, That funds available in this section for the Youth Mentoring Grants Program may be available for transfer to the Department of Justice Youth Mentoring Grants Program.

…it was sexist when I got here

I find the concept of feminization—how the presence or predominance of women in certain roles or occupations affect those roles and occupations, their legitimacy, compensation, etc.—to be fascinating and directly affect areas I work in (nonprofits, service, social media). Below is the abstract from a paper presented to the American Sociological Association by Paula England, Paul Allison, Yuxiao Wu, and Mary Ross entitled “Does Bad Pay Cause Occupations to Feminize, Does Feminization Reduce Pay, and How Can We Tell with Longitudinal Data?” (2004):

Predominantly female occupations pay less than “male” jobs, even after adjusting for skill demands. The devaluation perspective sees sex composition to affect wages; it says that gender bias affects employers’ decisions about the relative pay of “male” and “female” jobs. The queuing or relative-attractiveness view sees occupations’ sex composition to be affected by their reward level, with less attractive jobs going to women because employers prefer men and can get them in jobs that pay well. Past longitudinal research on how changes in occupations pay and sex composition are related has employed the cross-lagged panel (lagged-Y- regressor) model, generally finding support for the devaluation but not the queuing/relative attractiveness view. We argue that a stronger statistical approach to assessing causal dynamics is a fixed-effects model with lagged independent variables. Using CPS data from 1983 to 2001, we test these two perspectives. We find support for neither idea. That is, generally, the feminization of occupations does not lower their wages, and a fall in occupations’ relative wages does not lead to feminization. We conclude that in earlier historical processes, as occupations and organizations originate, there was a causal relationship between pay and sex composition, but that the continuing relationship is due to institutional inertia freezing in that early relationship, rather than to ongoing causal dynamics.

Emphasis mine.

A form from my favorites

Above is the signup form from Brompt, the blog reminder service I built a few years ago for undisciplined bloggers like myself. I’m very proud of this form: it’s clear, it contextualizes the data being requested within the functionality of the application, and it drives people to action.

There are a few areas for improvement: asking for your RSS feed rather than your blog’s URL is a bit roundabout (autodetection is still in the feature parking lot) and it could use a little explanation at the top in case you bypassed the front page’s explanation of the service.

Another reason I’m proud of this form is that it was built using Drupal’s Forms API—and Drupal uses awful, unintuitive forms like they were going out of style.

Towards advocacy-based media

Writing about Survival News yesterday, it behooves me to quote from Francine Adkins-Alexander’s “Progressive media’s wrong turn: Adversaries vs. Advocates”:

It would seem progressive media has missed a tremendous opportunity by taking up the adversarial format. Even though individual programs have enjoyed great success and have succeeded to a large extent in countering much of the vitriol and misinformation coming from corporate-funded conservative talk radio, they have not improved the media landscape. If anything, the [adversarial] “progressive format” has overshadowed those few outlets that dig for unreported stories, while presenting itself as one of those very outlets. It has, to a large extent, replaced advocacy with sensationalism and front page hysteria. It has made progressive media bigger and shinier — but not better.

I would like to think we will see the advent of yet another format geared to the progressive community, one that emphasizes advocacy. The advantage to this format is that it is not angry and it doesn’t need to blame, meaning that it can attract NPR audiences that avoid adversarial formats. It focuses on issues and how to solve problems. It can keep an audience energized by bringing to light situations that may be ignored by the mainstream media. And it can give audiences a sense of purpose. There is certainly room for anger when it comes to our social ills, but our reaction should not stop there. Progressive media was headed in that direction before it was co-opted by a few big guns that used conservative media as their blue print [e.g. Air America Radio, the Huffington Post] . Progressives are about progress, making changes, getting things done. The conservative format is not optimal for that kind of engagement.

The best journalistic infrastructure is still found at the larger news agencies, like the Times and the Post. If a key story is going to be broken, it will likely be broken at one of the big shops. And it will be highlighted or buried there. Progressive news aggregators serve an essential purpose in making sure we know the non-corporate storyline. I’d like to see progressive media even stronger in that role, combined with a broadcast format that keeps people involved as well as informed. That will require a shift in framework and media style. I hope someone, somewhere, is working on this approach.

Advocacy in print - Survival News for 2011

Today I sent another issue of Survival News—“the voices of low-income women”—to the printers; this is my second year as layout artist for the newspaper. At 40 broadsheets (11”x17”), the newspaper is the same size as last year. Published by Survivors, Inc., Survival News is the official newspaper of the National Welfare Rights Union and provides news, stories, advocacy information and 20 pages of Survival Tips—explanations and contact information to help in navigating the complex and crumbling web of social services and support systems—in English and Spanish, and excerpts in Vietnamese.

It’s a great group with which to volunteer and the stories and information that’s published are invaluable. Also, the group doesn’t just stop at publishing the paper; they go into DTA (Department of Transitional Assistance: welfare) offices and volunteer as public advocates for those seeking assistance. This is an excerpt from a DTA Log by Diana Moon:

I started working for Survivor’s Inc. early last 2010. I met many wonderful families and individuals. Thanks to all for sharing their experience with me. Here are a few highlights that I recall from my visits to Dudley and the Revere DTA.

I offered “Survival News” to a woman explaining a little about Survivor’s Inc and also introducing myself. She then told me her story. She worked for many years and had good jobs and was self-sufficient. Last year she became ill with heart disease and was hospitalized. Unable to work she lost her apartment and moved to her daughter’s apartment. She is on a wait-list for housing. She is in today because her disability benefits were terminated because she has a burial insurance policy (which is counted as an asset.) At first she thought surely this must be a mistake; I agreed with her that a burial policy which can only be used at the time of her death should not be counted as an asset (WT**.) Wow… how perplexing! Then she became emotional because after all the years she has worked and all she’s been through it’s really painful to be spoken to by a DTA worker as if she has done something wrong; as if needing financial help or any other assistance or getting sick means a failure on her part. I felt upset too for this very dignified woman to have someone speak so carelessly to her. We must keep trying to be respectful to each other. We reviewed the Survivor Tip section and I encouraged her to contact Greater Boston Legal Services (GBLS). She knew of GBLS and said she would contact them although she informed me she learned there is a Regulation that burial policies count as assets. I hope this regulation will be changed! She thanked me for the paper and after we talked she said she felt a little better and less alone.

I talked with another woman about the Survival Tips section and we had a discussion of private rights and personal space and then what can happen to privacy when you live in public housing. For example, there is talk of making smoking illegal in public housing (more issues to debate). She told me she came in to apply for food stamps. She was laid off her job and then took a position for less income at a CVS. The food stamps are a big help she said. I’ve heard similar stories from other women who either work full time for minimum wage or have a part-time position and come to apply for or renew their food stamps. Food stamps are a great benefit and I feel thankful this is available for families! I think, too, about how many live vulnerably just one paycheck away from the threat of losing the security of their home. Another woman told me, “If I lose my job then I’ll be looking for shelter. I worry each month I won’t be able to make full rent and then I’ll be evicted.” I think about how we hear talk about strengthening families, about empowering families; well, then affordable, safe housing is a good place to start!

The artwork above was contributed by coworker Billy Brown. And I’m still trying to convince them to work on their website.

The 48 hour mobile web app: Drunken Stumble

Last weekend I participated in the Boston Hack Day Challenge, a 48 hour (so I’m not sure why they called it a hack day) competition sponsored by the Boston Globe and held in the MassChallenge workspace. The goal of the event was to develop tools that would improve the lives of Bostonians. My team won “Best Mobile App” as well as the “Crowd Favorite” award for the pub crawl app we built: Drunken Stumble.

Update: You can also read about the process from some of my teammates: @nikibrown on the design, @unruthless on frontend and whip-cracking, and @mikemiles86 on the backend and interface.

(And if you’re wondering how a pub crawl app would improve the lives of Bostonians, you must not be from Boston.)

About the Project

Our team formed Friday night. On the frontend we had @nikibrown (designer) and @unruthless (frontend developer)—who knew each other prior to the event. On the backend we had @mikemiles86 (application developer) and myself (API and interface developer)—who happened to be standing near the cheese table.

Together—and with the muse of the open bar—we came up with the idea of creating an application that would help people perform a pub crawl. While other teams proposed concept demos and APIs, we set our goal to be the delivery of a complete product by the end of the weekend. Defining a minimum viable product meant keeping a lot of great ideas in the parking lot (like drink lists and multi-user crawls) but helped us focus on delivering a tight, attractive and functional pub-crawl application that lets you:

  • use your smartphone to find nearby pubs

  • get walking directions to that pub

  • invite your friends to the pub via social media (Twitter and Facebook)

  • find the next pub for your crawl and get directions to there from your current pub

  • track your nightly progress (or revisit it the next morning)

  • call a cab home at the end of the night

About the App

We built Drunken Stumble as an HTML5 mobile web application—testing it on both iPhone and Android. As an HTML5 mobile web app, we can access smartphone features (like GPS and Portrait/Landscape modes) as well as quickly write, test and deploy. Both @mikemiles86 and myself are Drupal/PHP developers so (with some reservations) we decided to use Drupal as a framework—using its paths, forms, database and template systems and nothing else.

Planning documents

I focused most of my development on interfacing with external APIs. Originally we planned on using Yelp to provide business data for locating pubs, but their API is extremely limited with only 100 lookups per day. Fortunately, we found an awesome service called SimpleGeo that currently offers unlimited lookups for businesses based upon location. Using SimpleGeo we were able to quickly write a rich, location-aware application that works anywhere in the United States, not just in Boston.

Once we knew we had a source for business data, the next step was telling the application where you are. HTML5 offers native geolocation which means we can (politely) request exact location data based on your smartphone’s GPS. If your smartphone doesn’t have GPS (or it’s turned off) you can type in an address and we use Google Maps to geocode it into a latitude/longitude. If you do use your GPS, we also do a reverse geocode (again using Google Maps) to show you your street-level address to confirm that’s actually where you are (“ 42.331528,-70.94425” could be in the middle of Boston Harbor for all I know).

Google Directions is the special sauce that ties it all together. We provide your starting location and the location of your next pub and Google Directions provides walking instructions and waypoints along the way. We display the walking directions and overlay the waypoints on top of a (static) Google Map. Unfortunately, Google Direction’s API also has a rather low limit of lookups (2,500/day), so should we go over our limit (as we did about 2am Sunday morning), we alternatively display a link that will launch your smartphone’s native maps app (or push you to Google Maps directly) to get directions there.

There was one last benefit of using SimpleGeo’s well-populated business database: our app’s design isn’t limited to just pubs. For example, we decided rather late in the process to offer a list of local taxicab phone numbers at the end of the pub crawl. Because we had already tapped into the SimpleGeo API, we just needed to filter nearby businesses for “taxis”, rather than “bars & pubs”. Which is an important thing to keep in mind: with a few minor changes, our app can facilitate routing for any type of business or geographically based event—like a taco crawl or artists’ open studios.

About the process

The process was awesome. We got down to business about 8:00pm on Friday night and went live at 1:45pm on Sunday. Sure, we missed deadlines, our final feature list shrunk (and the parking lot grew), the architecture is far from “robust”, and the final design didn’t match our initial sketches (it was better!), but we met our goal of delivering a complete and functional product.

I think it was our focus on “completeness” that helped us win more awards than any other team (2 awards). Other than a few all-hands decisions, our small and diverse team focused separately on our individual areas and integrated as necessary—@mikemiles86 and I used Git to sync progress on the backend, while I think the frontend mostly looked over each other’s shoulders. For me that meant moving from API wrangling at the start of the project to templating at the end. I also can’t overstate the value of @mikemiles86 pulling an all nighter on Saturday (the rest of us ejected at 3am for about 4 hours of sleep)

The Crowd Favorite award was not only a function of our easy-to-understand concept, but also the stellar work of our frontend team. @nikibrown quickly produced strong branding and interface mockups that were key to creating early buzz: as other teams walked around and mingled, we could easily show off our idea. @unruthless became our de facto project manager and spokeswomen, explaining the app to visitors and pushing progress updates to Twitter and the #bostonhack hashtag for the event. We also lived our values, bringing in a wide selection of beer to carry us through Saturday and Sunday (and sharing it didn’t hurt our chances of winning the Crowd-Favorite award either).

About the future

It’s only been a few days since the event but we’ve kept up a steady stream of chatter over Twitter and pushed a few minor updates to the server too. The intensive hack model worked really well for building a minimally viable product, but time will tell whether we can keep the momentum and updates coming—turning Drunken Stumble into a maximally functional application.

In the meantime though, happy stumbling, sober or otherwise.

How to decorate your office without really trying

Tonight I watched the movie musical How to Succeed in Business Without Really Trying. I was shocked, though not surprised, to discover that all of the office furniture from this 1967 film is the same furniture I have in my office at UMass Boston. The scene above is more representative—the desk and shelving are the same (and I used to have that typewriter stand)—but the orange chair in the scene below is mine.