Menu

Sanspoint.

Essays on Technology and Culture

A Brief Thought on watchOS 3 and Context Aware Computing

A key feature of watchOS 3, as announced at WWDC, is that it’s now easier to configure and switch between watch faces. This is awesome, and a lot of Apple Watch users, myself included, have several watch faces they switch to at different times. Currently, I keep an “Activity Face”, a “Productivity Face”, a “Sleep” face, a “Casual” face, and a “Time Only” face. Being able to have them a swipe away, instead of a force-press and a swipe away will make them even more useful.

Daniel Jalkut, however, had a better thought: using Siri to switch between named watch faces. This is brilliant, and I hope Apple considers his radar. If they decide to do this, if not in watchOS 3, perhaps in 3.1, it would go a long way towards something I dreamed of when the Watch first dropped: context-aware watch faces.

As long as the Watch is tethered to the iPhone, something I don’t see changing any time soon, it knows where I’m at and what I’m doing. If I’m walking to work, why not let it switch to my Activity face, so I can see my ring progress? When I’m at the office, show my productivity face, so I get my OmniFocus tasks. At home, show my casual face. Bedtime? Switch to the Sleep face. Weekend? X-Large face, please.

Yeah, it’s a bit of a pipe-dream, but as long as the data around knowing what my Watch face should show stays securely on my devices, I’m more than happy to let it be used to make my life a ittle more convenient. And yeah, stick the Siri integration on there, too. That would be great when you have speciality faces that don’t necessarily fit context, or if you’re paranoid about location data.

I want my devices to make my life easier, to be my outboard brain. That requires them to know more and step up what they can do for me. I’m more than willing to allow it, as long as that data remains in my control, and not sold to the highest bidder. The WWDC announcements this year have left me hopeful, more so than iOS 9’s weak-sauce Proactive stuff, but every little bit helps. You don’t need massive buildings of data-sucking machines in the cloud, you just need to use what’s already in people’s pockets, and on people’s wrists in a smarter manner.

Am I Too Paranoid About AIs?

Some of the tech people I follow online are slowly starting to crack my brain open about bots and AI stuff. John Gruber and Merlin Mann on The Talk Show were quite effective. There’s a lot of possibility, and as an Apple fanboy, I’m excited to see what’s happening with Siri at WWDC. In the interim, the Amazon Echo continues to tempt me, despite my misgivings. I’ve been a proponent of context-based computing for the past few years, and with better bots and AI, we seem to be getting there, at last.

Problem is, to get all those crazy cool context-aware systems, for our AIs to know what we need to know before we need to know it, they need a lot of data about us to make it happen. I don’t want to give all that data up to those systems. It’s less that I’m worried about giving up my personal data in the abstract. I’m more worried about what the people I give my data to are doing with it beyond what I want them to do. It’s a question of trust. Am I giving up more than I’m getting back?

To stick with Google, my experiences with their AI stuff have been sub-par to say the least, but I don’t know why. It’s a black box. Maybe I wasn’t giving Google enough data, maybe I was stymied by iOS limitations, or maybe I was some weird edge case. There’s no good way to diagnose where any of this stuff is failing, and—at the time—no easy way to make corrections when the AIs screw up. Why should I trust Google to know my commute, when it gives me reminders to leave for work once I’m already at the office?

iOS 9’s “Proactive” features got me more excited than anything Google’s done, not least because I knew most of the smarts were happening on the device, instead of the “cloud” where Apple could do squirrely things with the data. I trust Apple in a way I don’t with Google, but Proactive is a disappointment. Maybe there will be improvements with iOS 10, but even for a 1.0, Proactive is weak sauce. The most functional thing it does is show a little corner icon on my lock screen to open Overcast when my Bluetooth headphones connect. This does me no good, because I still hit the home button to view my lock screen like an animal, and since I have a 6S, I end up at my home screen.

So, I’m stuck between a service that barely works, and a service that might work if I’m willing to unload my entire digital life into its hungry, gaping maw. I know Google will use that data to give me something, but then they’ll slice it, dice it, mix it with the data of people it considers similar, and sell it as a package to advertisers. That’s how they make the money to keep the services going. We know that this is the deal, but the question is… should I really be that paranoid?

It’s a tricky question. How do I know what I’m missing out on until I try it? But I can’t try it without going all-in and surrendering my personal data to a service I don’t know if I can trust. As mentioned before, my previous experience with Google’s AI stuff have been phenomenally sub-par for reasons I can’t even begin to unpack. If they want me to go all in, they’ve got to give me a compelling argument to overlook where they’ve failed in the past. Google not only needs to overcome my paranoia, but to overcome their own failures.

My paranoia extends far beyond Google, though. I’ve made it a point not to connect anything with my Facebook account, because as little as I trust Google, I trust Facebook even less. I even disabled the ability to use Facebook with apps. That hasn’t stopped Facebook from figuring out pieces of my digital life I thought had been siloed. I’ve seen Facebook suggest Twitter friends as Facebook friends, and all I can think of is “How did they get that?” Then, I realized I linked my Instagram account, which uses the same email as my Facebook account, to Twitter, because I was unhappy with IFTTT over their poor treatment of Pinboard. That one’s on me, I guess.

The question remains. Even when I think I’ve drawn the barriers between myself and the prying eyes of the algorithm, something always leaks. You think you’re safe, and then the algorithm starts showing you stuff you never knew it was going to give you—correct stuff, but not the right stuff. The only way to correct it is to surrender, give up more data, and surrender more of myself as disembodied data points that will get sold to give me more and more “relevant” ads. It’s a Catch–22! I don’t want to have my data sold, but I want at least some of what the AI algorithms can give me.

I’m not a hard-liner on any of this, I just want to know what I’m giving up, and how it’ll be used to serve me, and their real customers. At least then I can make an informed decision. If I buy an Echo, can I be certain Amazon is really deleting anything I would say in my apartment before “Alexa?” Am I going to get ads for walnuts based on causal conversations with my fiancé about nutrition? I mean, I’m the kind of person who will give fake information when signing up for a store loyalty cards so I don’t get more junk mail and telemarketing calls. I can’t do that with bots and algorithms.

How long can I keep putting up the fight? At a certain point, it’s easier just to give in. My only hope is that I can hold out until the adtech bubble finally bursts. At which point, I might have to pay a monthly fee to get a decent AI system in my life, but I’ll be more comfortable that way. Either that, or Siri will get the long overdue upgrade it needs at WWDC ’16. There’s no rush, but that doesn’t mean I’m comfortable being left behind.

Mindful Tech, Part 8: Where Our Data Lives

Of all the ways in which we use technology, nothing has changed quite so dramatically as the way we store and retrieve data. In the span since I started using a computer, we’ve gone from keeping our files on floppy disks and cartridge media, to CD-Rs and thumb drives, to having more local storage than we know what to do with, to keeping our data in the cloud. Odds are, we probably have some overlapping mix of all of these, save for the floppy disks unless you’re really old.

This is just data you’ve made for yourself: files, photos, music, and anything else you can imagine. Where does all of this live? Is there one place you can point toand say “my data lives here”? Another thing that’s changed is that we now have multiple devices. Maybe it’s just a computer and a smartphone, but many people also have tablets, separate work computers, network-attached storage…

I remember when I started living with more than one computer in my life. I was obsessed with keeping everything in my digital life in sync across all my devices, and having one single place for all my data. There were a number of adventures in doing both, never successfully. This included destroying the ability to use .Mac synchronization on two computers thanks to a pirated, third-party, unsupported app to trick computers into using a locally emulated .Mac sync just so I could have all my Yojimbo notes on my laptop and my desktop, years before Evernote was a glint in someone’s eye.

A decade later, and there’s no shortage of great options for keeping data in sync across multiple devices. It’s a blessing and a curse all the same. I know I have personal data scattered across iCloud, Google Drive, Dropbox, and probably a few others. Everything is in sync (I hope), but it’s far from being in one place. It would be wonderful for all of these storage options to work together in some fashion. If there were an app that lets me see, at a glance, all my data across my various cloud storage providers, I would pay handsomely for a copy.

There’s no incentive for the cloud data providers to allow such a thing, of course. It’s more lucrative to keep you locked in with exclusive features and APIs, and—depending on who you’re storing with—to pry into your data for the purpose of getting more info to sell to advertisers. There’s also the matter of trust, a subject I’ve written about elsewhere. There are plenty of people who stand by Dropbox, and it’s a reliable service. With Dr. Condoleeza Rice, a noted proponent of domestic spying, on the Dropbox board, I’d rather keep my data some place else. I’m forced to keep using Dropbox, however, as it has the most robust sharing options. iCloud Drive is reliable, but accessible only to me.

Yet, as our devices proliferate, and the way we use computers changes in tune, our data is going to be forced to live in the cloud. It’s unreasonable to expect people to set up their own cloud storage in their homes with a NAS or spare computer. It’s expensive, requires time for setup and maintenance! and ISPs still frown at users running home servers. It’s easier now than it was when I took an old PC and used it as a file server for my MP3s over a decade ago, but not easier enough. It’s not something we should expect ordinary people to do.

Let’s go back to the great auditing from earlier in this series. If you’re concerned about how and where your data lives, take the time to audit that. Ask yourself what data needs to even be in the cloud, what service gives you the best balance of accessibility, organization, flexibility, and security—especially security if you’re as paranoid as I am. Settle on a primary choice, and a backup solution to cover what your primary storage provider can’t do. It makes life so much easier when you know what data lives where.

And if you’re a developer who’s good with APIs and data visualization, please make an app that lets me see all my files in the cloud in one place. Mac or iOS, I don’t care, just make it, and charge me for it. I doubt I’m the only person who wants it.

Thoughts on the Coming Chatbot Revolution

Well, folks, the tech press and VC establishment have shaken their Magic Eight-Ball and determined the Next Big Thing is… “Bots!”

Wait. Is this right? Grace, can you check on that? Really?! Okay, I’ll roll with it.

Yes, it’s bots. Specifically chat bots and AI virtual asistants like Siri and Alexa. Everybody’s getting on board with the bot revolution, and it’s going to revolutionize everything. Get your VC investments primed and ready for all the bot statups.

The rise of the “bot” as the Next Big Thing from the Valley utterly mystified me until recently. What are the advantages of a conversational interface over an explicit, directly manipulable one? There’s the hands-free aspect, something I’ve appreciated with Siri, even more on my Apple Watch, but that only works with the voice assistants. Chatbots? Not so much, though we have come a long way since the days of “YOU CAN’T GET YE FLASK”.

Then it hit me, in that way so many things do. Chatbots, especially when they have a playful personality, are a perfect way to extract more data from people. With Internet users becoming more mindful of their privacy, it’s getting harder for the data brokers and ad companies to get more info to sell advertisements on. What better way to learn consumer preferences than by having them give it to you directly? No more inferring user interests from cookies and browsing data! By presenting a conversational interface, you bypass the defenses of a user’s protectiveness, and get a direct tap into their needs and wants. No wonder it’s a growth industry.

Bots and AI seem like a useful solution being applied to the wrong set of problems. There are great applications for these tools. If I could sit here, at my desk, and be able to just capture a quick idea or OmniFocus task by yelling out loud to my Virtual Assistant, that would be great. I mean, I can… but it’s not great. The chatbot paradigm has the advantage of being simpler than a GUI, and for a number of simpler tasks, it should be a lot easier than one. But it won’t be for everything.

Anything that involves dealing large amounts of data is going to be worse. If you’re looking up pizza places, you’re already going to be overwhelmed in some New York neighborhoods. Instead of finding better ways to handle that data, you’re likely to just be defaulted to whatever chain pizza joint has a marketing deal with your bot provider. Hope you like Dominos, is all I’m saying. Because of this, visual, tactile, and direct interfaces will never go away. Even the voice-controlled universe on Star Trek, where the computer never has trouble understanding you, has a GUI all over the place.

It’s possible a conversational UI would help in allowing computers to be better at understanding nuance. Historically, this is something computers have always sucked at. This is, of course, based on the assumption that the people creating the bots have an understanding of nuance as well. Alyx Baldwin wrote a great piece on The Hidden Dangers of AI for Queer and Trans People. It’s worth your time, but here’s a summary: computers are really good at putting things in boxes, and humans are really bad at being put into boxes. The people who program computers are also lazy and tend to only think of a handful of boxes. Unless the developers of AI, Deep Learning Algorithms, and Chatbots understand the variety of people using them, the AIs, Algorithms, and Chatbots won’t understand them either.

As for understanding, even in terms of language, that’s still up in the air. Voice recognition has come a long way, and on a good day, Siri can understand me despite a whole mess of background noise. Voice recognition still sucks, however, for anyone who speaks with a heavy accent, or has a speech disorder. Since bots and voice recognition systems are often trained on a corpus of speech that assumes someone who speaks a standard language by default.

If you’re not going to come across that language or method of speech in a Silicon Valley development house, you’re not going to see it supported in a voice recognition app or device. It is possible to do single-user training, much like you would with old school voice-to-text apps like Dragon Dictate, but that’s a lot to ask of a user up-front. Easier to just let ’em dangle, though perhaps that might change in time.

Unlike, say, virtual reality, I can see a lot of potential in the “bot” ecosystem, assuming we can work past all these stumbling blocks in the way. I’ve eyed an Amazon Echo for a while, though its utility would be diminished since I refuse to use any streaming music service. We’ll see what happens after WWDC, there. I’m still uncomfortable letting Amazon have an always-on microphone in my apartment, if only because I can’t be sure it’s not going to be parsing my conversations for ad metadata. I could be more willing to trust an Apple device, even if it does less, because Apple is more in tune with me on privacy.

The dream of the AI/chatbot/virtual assistant world is one where everyone’s little earpieces, smartwatches, speaker dinguses, or whatever, seamlessly connects the entire world by our voice, enabling an easier lifestyle for everyone. The reality is likely to be a whole bunch of miserable walled gardens full of microphones that can deliver us crappy pizza while making sure we get ads about debt consolidation every time we complain about the credit card bill after buying one. The former is more preferable, but the later is much more lucrative.

Mindful Tech, Part 7.5: Our Data Trails, Ourselves Continued

Before you read this, take a moment, and check out Take This Lollipop. Fair warning, the site requires Flash, and it needs to connect with your Facebook account to work. It’s worth trying, at least once, and you can always disable its Facebook access when you’re done watching.

Go ahead. I’ll be here when you’re done.


Take This Lollipop is creepy, and a bit heavy handed, yet it makes a point about who has access to your data. It also reveals the potential of our data to create narratives. In a world in which our data is constantly being used to create a specific narrative for us, e.g. you’re a White Male, age 18–35, with an interest in Consumer Technology, and 80s Music, who is also $36,000 in debt, so here are Relevant Advertisements—we have the power to use our data trails to create narratives about ourselves as well.

Recently, I had the pleasure of seeing a talk by Lam Thuy Vo, a Data Journalist and Data Artists, at Facets 2016. She showed off a series of personal projects that used data to examine the very human lives of herself an others. These include Quantified Breakup, which examined her own data on movement, messaging, finances, and more, in the wake of her divorce. It’s a fascinating and different way of thinking about data, and a great contrast to the almost paranoiac view in the previous Mindful Tech piece. She also introduced us to Take This Lollipop, as well.

Data trails are more than just what’s collected for advertising purposes. We collect data on ourselves, deliberately and not-so-deliberately, and in ways we don’t even think about. If you wear a fitness tracker, you’re collecting data on yourself deliberately. If you carry an iPhone, you have a record of everywhere you go, not so deliberately. Data trails encompass the thoughts we post to Twitter, the emails we send on Gmail, our browser histories, the music we listen to on Spotify, anything we do online, for better and for worse.

It’s becoming more and more impossible not to opt-in to even some of the most egregious data collection. For example, when I was looking for work, I discovered pretty quickly that if I don’t have a LinkedIn profile, as far as most employers were concerned, I did not exist. This may not be an issue if you’re working in manual labor fields, but if you want a desk job where you’re moving data around, if you’re not on LinkedIn, you don’t exist. When all of your friends and family are on Facebook, and you’re not, how does this change your social landscape in the real world? And, of course, what happens if you’re blocked from one of these networks for whatever reason? [1]

There’s no clear answers here. Lam brought up the idea of a Digital Bill of Rights that determines who has the right to our data and when. There’s a social difference between attitudes to data privacy between the United States and other parts of the world. You run into ideas like the Right to be Forgotten in Europe, but when the Internet is dominated by American corporations with American ideas of privacy and data retention, attempts to legislate our way out of this are doomed to be insufficient.

In the interim, the best option is to learn about your data, and to take ownership of it. Ownership of data matters. One thing that Lam pointed out in her talk is that it is possible to pull your data out of many of these services. Whether it’s human-readable is another matter. The best you can typically hope for are CSV files, which you can manipulate using the most humble of data analysis tools: Microsoft Excel and PivotTables. It’s then up to the viewer to create a cohesive narrative from that data: a story with a beginning, middle, and end.

A while back, I wrote about how I want to know what the services I use know about me.

“If I shouldn’t worry about the data I feed to Google, Facebook, and a whole holy host of similar companies and services out there, why not be more transparent about what data is being collected, how, and what they know about me? I want to see a simple, clean, human readable page on every service I feed my personal data to that tells me every last piece of information that they know…”

There’s an opening for services that can do this for people, though the privacy risks of aggregating all this data together are significant. If a malicious actor gets in to a service that houses the aggregation of all of our personal data, it’s not hard to see the potential for abuse. It would be a revolution in doxing alone. Instead, I’d like to see tools that exist in the user space, off the cloud, that let us analyze and identify the stories in our data. The better to know what we’re making, what we’re leaking, and what we should be deleting.

And even deleting our data is problematic. The database design of many websites is such where it is easier to mark a database record as inactive, rather than remove it entirely. This is one part lazy design, and one part technical limitation. How can we be sure that the data we’ve deleted is truly gone, when we want it gone? What happens when the data trails we thought were lost when a service dies get bought by another company? Truth is we don’t know. And that makes thinking about it all the more essential.


  1. This is huge. Facebook’s “Real Name” policy has had a chilling effect on transgender people, or anyone who needs a pseudonym to avoid harassment and abuse, locking them away from digital support networks, family, and friends.  ↩