The Knowledge House, JavaScript’s d3, reflections on scalability / complexity

It’s been awhile! I’ve been doing curriculum development and tutoring for The Knowledge House, a Black-owned bootcamp serving the Bronx community that is awesome and y’all should check it out. They operate outside of “working hours” to allow students to simultaneously work full-time, cover travel, offer a stipend, and can help students with food and housing security.

Here’s their mission statement:

“The Knowledge House empowers and sustains a talent pipeline of technologists, entrepreneurs, and digital leaders, who will uplift their communities out of poverty.

We are a social change organization. We passionately believe that young people should be given the opportunity to succeed regardless of their financial circumstances or ethnic background. We educate, empower, and mentor New York City residents, from low-income communities with the technology and workplace skills needed to launch a successful career in the technology industry.

Through our digital skills programs, The Knowledge House is transforming the tech industry by building an army of talented workers equipped with the latest skills needed in today’s economy.”

Meanwhile, in my personal life…

I’ve been trying for a long time to figure out how to best render my network of rampage citations, and I’ve gone through a LOT of “failed” ideas. Maybe someday I’ll write a post detailing those trials and tribulations. The end result wound up being that the only thing I can find that does all the things I want is a JavaScript library called d3. This meant that I first needed to learn some JavaScript, which was fine. I also want to have my graph connected to my database so I don’t have to go back and recreate the graph every time another rampage happens (kind of a macabre concern, but there you have it), and to be able to put information into the database in a streamlined way– heck, maybe even enable other people to submit information. (Here’s a little featureless mock-up I did in d3 without knowing d3 [some of this stuff I understand by the time of this entry; some I just managed to cobble together looking at examples], just connected to some static trial data– the stuff we had so far in the other posts. Not fancy, but you can at least see what the code tends to look like.) And I want this to be scrapeable by the general public.

Previously I’ve had an issue where I haven’t thought ahead enough about whether the technology I’m using is going to expand well to include my end-goal features. I’ve been fooling around with various aspects of this project for a long time, since I could barely code. Back then it was more important for me to motivate myself through achievable subgoals, and I didn’t have the skills / programming maturity / vocabulary to even be able to find out what those features would entail concretely, and what packages or languages that would require, and so on– trying to design the whole thing and then fill it in piecewise like a color-by-numbers was not a viable strategy. I still don’t think I’m necessarily in a place to design the whole thing start to finish and limit myself to what I know of now, but I am at a point where I can start thinking a few more steps ahead at a time than I was before. Yet at the same time I’ll probably have to do each major step in an inferior “practice” version before I learn enough about the general category to optimize.

Case in point, because of my working at TKN (described above) I’ve been exposed to people building Flask apps, and that has a lot of the components I’m seeking. I can use SQLAlchemy to hook it up to a database, use a POST method to let people (including myself) submit stuff to the database, output everything via API, and hopefully hook up a nice d3 graph. I don’t even know about the other packages/libraries for this kind of thing, but right now I think it will be best for my learning process if I just go on ahead with this then backtrack and learn something else if it turns out it’s missing a crucial aspect. I’ve been working on one using failed/aborted/prevented school shooting attempts scraped from the same Wikipedia page as my trial data. (I have some complaints about what’s included on the list and what’s not– namely Sol Pais, whom I don’t believe was ever going to attempt harm to anyone before a federal manhunt led to her private suicide– but I’ll worry about my Wikipedia contribution duties later.)

I’ve dragged a d3-fiend friend on board my d3 effort by offering to pay him per-text to be my on-call tech support / Q&A. (It’s not about the money, he’s just kind and wants to help someone learn, and believes he’s more likely to be diligent and prompt if he feels accountable because he’s getting paid.) I find this a charming arrangement. And historically, having to explain the whole stuck point out to someone often results in my solving the problem myself before I even hit send, so it may turn out to be more of a rubber duck situation. We’ll see!

In the next post: Flask basics; enough Heroku to, and ONLY to, get a Flask app live; Jinja.