unique hazards may exist https://uniquehazards.com/ en Sat, 16 Sep 2023 08:59:09 -0700 https://uniquehazards.com/2023/09/16/the-future-is.html Sat, 16 Sep 2023 08:59:09 -0700 http://stlhood.micro.blog/2023/09/16/the-future-is.html <p>The future is gradually getting a little better.</p> <img src="https://cdn.uploads.micro.blog/78093/2023/image.jpg" width="450" height="600" alt=""> Emergence is all you need https://uniquehazards.com/2023/06/09/emergence-is-all.html Mon, 12 Jun 2023 09:22:52 -0700 http://stlhood.micro.blog/2023/06/09/emergence-is-all.html <img src="https://cdn.uploads.micro.blog/78093/2023/5fb7fad3e4.jpg" width="600" height="399" alt=""> <br/><br/> <p>I recently spent a couple of weeks building-out a LLM-driven product concept (which I’ll be writing more about soon over on Mozilla’s blog). The experience triggered a lot of thoughts and feelings. Among them is the realization that the way we build software products is about to change irrevocably.</p> <p>For as long as people have been writing software, it’s been a largely predictable undertaking. Someone like a product owner figures out what the product should do. Someone like a designer develops that definition into a user experience. And someone like an engineer turns it all into functioning software.</p> <p>The resulting product is usually deterministic. It does what it is supposed to do and behaves as specified. When it doesn’t, we call that a <strong>bug</strong>.</p> <p>Products built around AI are different because their behaviors aren’t nearly as predictable. Each level of the solution stack requires you to make decisions — e.g. your choice of model, your approach to prompt engineering, the settings and characteristics of your runtime environment, the way your interface manifests both the user’s inputs and the model’s outputs — and those decisions then interact with each other. You’re building a complex system that is more than the sum of its parts.</p> <p>The resulting product isn’t deterministic, it’s <strong>emergent</strong>. It doesn’t behave the way you expected, only now that’s not a bug, it’s a <strong>feature</strong>.</p> <p>In this new world, the traditional tools and mindsets of product development fail us. Writing a detailed spec or designing a fully-realized experience doesn’t make sense in a world ruled by emergence because you simply don’t have the same degree of prediction and control. Mapping out a shipping schedule becomes an exercise in futility when every build your team puts out can exhibit wild new behaviors that impact both your thinking and your planning.</p> <p>We’re no longer professionals working from a manual. We’re kids playing with home chemistry kits, mixing random vials of liquid and waiting for a puff of smoke. We’re in a new universe and we don’t know the physics yet.</p> <p>It’s all rather exciting. <br/><br/><br/></p> <p><em>(Image credit: Airwolfhound, CC BY-SA 2.0 <a href="https://creativecommons.org/licenses/by-sa/2.0">https://creativecommons.org/licenses/by-sa/2.0</a>, via Wikimedia Commons)</em></p> Running Vicuna locally: the complete idiot's guide to basically running ChatGPT on your own laptop for free https://uniquehazards.com/2023/05/06/the-complete-idiots.html Sat, 06 May 2023 21:47:34 -0700 http://stlhood.micro.blog/2023/05/06/the-complete-idiots.html <p> <div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;"> <iframe src="https://www.youtube.com/embed/YWgkVXLSWt0" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" allowfullscreen title="YouTube Video"></iframe> </div> <br></p> <p>The new <a href="https://lmsys.org/blog/2023-03-30-vicuna/">Vicuna-13B large language model</a> is a Very Big Deal. A big enough deal to <a href="https://simonwillison.net/2023/May/4/no-moat/">drive one of the biggest players in AI into a state of panic</a>. Why? Because it produces results of near-ChatGPT quality with a model that&rsquo;s a fraction of the size, is freely available, and runs comfortably <em>on consumer-grade laptops</em> instead of requiring a massive server farm.</p> <p>This feels like a flashbulb moment for AI, and I think it&rsquo;s important enough of a milestone that people should experience and contemplate it themselves first-hand. That goes for AI fanbois and doomers, alike.</p> <p>The good news is that you don&rsquo;t need to be an expert to try this for yourself. Thanks to the efforts of the open source community, even armchair amateurs like me can do it. All you need is a Mac with an Apple Silicon CPU (i.e. M1 or M2) and the willingness to copy and paste some commands.</p> <p>When you&rsquo;re done, you’ll have something that is nearly as good as ChatGPT, if noticeably slower. But it will be running <em>entirely locally</em>, on your own laptop, with complete privacy and freedom. Quite the feat.</p> <p><em>May 23, 2023 update: the instructions and download links below have been updated to reflect recent changes to llama.cpp, which require downloading a Vicuna weights file that is in a new format.</em></p> <h2 id="how-to-run-vicuna-on-your-macbook">How to run Vicuna on your MacBook</h2> <ol> <li> <p>Open Terminal.</p> </li> <li> <p>If you don’t have them already, install the Xcode command line tools.</p> <p>% <code>xcode-select --install</code></p> </li> <li> <p>Confirm that you have Python 3.10 installed.</p> <p>% <code>python3 --version</code></p> <p>If this command doesn’t return a version number (e.g. “Python 3.10.8”) follow these instructions to install Homebrew and Python3:</p> <p>% <code>/bin/bash -c &quot;$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)&quot;</code></p> <p>% <code>brew install python@3.10</code></p> </li> <li> <p>Clone llama.cpp from GitHub.</p> <p>% <code>git clone git@github.com:ggerganov/llama.cpp.git</code></p> </li> <li> <p>Build the app.</p> <p>% <code>cd llama.cpp</code></p> <p>% <code>make</code></p> <p>% <code>python3 -m pip install -r requirements.txt</code></p> </li> <li> <p><a href="https://huggingface.co/TheBloke/stable-vicuna-13B-GGML/resolve/main/stable-vicuna-13B.ggmlv3.q4_0.bin">Click here</a> to download the Vicuna model data file from HuggingFace (<strong>be warned: it’s a 7.3 GB file</strong>).</p> </li> <li> <p>Return to Terminal and move the model data file (”stable-vicuna-13B.ggmlv3.q4_0.bin”) to the necessary location.</p> <p>% <code>mv ~/Downloads/stable-vicuna-13B.ggmlv3.q4_0.bin ~/llama.cpp/models/</code></p> </li> <li> <p>Start the chat interface by issuing the following command.</p> <p>% <code>./main -t 8 -m ./models/stable-vicuna-13B.ggmlv3.q4_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 --interactive-first -r &quot;### Human:&quot; -p &quot;### Human:&quot;</code></p> <p><strong>IMPORTANT:</strong> Where you see <code>-t 8</code> above, change the <code>8</code> to match the number of CPU cores your Mac has:</p> <ul> <li>M1 = 8</li> <li>M1 Pro = 8 or 10</li> <li>M1 Max = 10</li> <li>M1 Ultra = 20</li> <li>M2 = 8</li> <li>M2 Pro = 10 or 12</li> <li>M2 Max = 30 or 38</li> </ul> </li> <li> <p>The model is now running. You should see:</p> <blockquote> <p>Running in interactive mode.</p> <p>Press Ctrl+C to interject at any time.</p> <p>Press Return to return control to LLaMa.</p> <p>If you want to submit another line, end your input in &lsquo;\&rsquo;.</p> <p>### Human:</p> </blockquote> </li> <li> <p>Just start typing, as if you were using ChatGPT! You’ll notice it is slower than ChatGPT, and there may often be a long pause before responses start appearing, depending on your prompt and your Mac’s CPU. But it should be usable.</p> </li> </ol> <p>(When you’re done and want to exit, hit Ctrl+C twice. To start the model again, just repeat step 8.)</p> <h3 id="example-output">Example output:</h3> <blockquote> <p>### Human: Write a 30-40 word backstory for a fictional character who is the marshal of a small settlement in post-apocalyptic Modesto, CA.</p> <p>### Assistant: James Heckleworth was once a police officer in Modesto, CA until the world ended. After his wife passed away from radiation exposure, he took up the role of marshal in the small settlement that had formed in the aftermath. Despite his grief and the constant danger of living in this new world, James has proven to be an effective leader and protector of his community. He is tough but fair, often showing compassion towards those in need while maintaining order and safety within the town&rsquo;s walls.</p> </blockquote> <p>Congratulations: you&rsquo;re using an AI that a year ago was the stuff of science fiction, and just weeks ago was under the near-exclusive control of big tech. Welcome to the (very weird) future.</p> <img src="https://cdn.uploads.micro.blog/78093/2023/fd8ea25c56.jpg" width="600" height="550" alt=""> <p><em>(Image: DALL-E)</em></p> https://uniquehazards.com/2023/04/12/picked-up-a.html Wed, 12 Apr 2023 15:33:59 -0700 http://stlhood.micro.blog/2023/04/12/picked-up-a.html <p>Picked up a <a href="https://stadlermade.com/">Stadler Made</a> outdoor wood-fired pizza oven. Made out of naturally-weathering corten steel, it arrives flat-packed and slots together fairly easily. I&rsquo;ve been able to get it up to a scorching 950F, which means proper Neapolitan pizza in 90 seconds. Ingenious design, great results, loads of fun.</p> <img src="https://cdn.uploads.micro.blog/78093/2023/2ba0cfa071.jpg" width="450" height="600" alt=""> <p></p> <img src="https://cdn.uploads.micro.blog/78093/2023/8e95145e90.jpg" width="600" height="450" alt=""> AI is an invasive species https://uniquehazards.com/2023/03/31/ai-is-an.html Fri, 31 Mar 2023 18:17:57 -0700 http://stlhood.micro.blog/2023/03/31/ai-is-an.html <img src="https://cdn.uploads.micro.blog/78093/2023/059902faba.jpg" width="512" height="512" alt="A robotic fish swimming in a river"> <p>Let&rsquo;s talk about carp.</p> <p>I ran across <a href="https://www.youtube.com/watch?v=t3oLeSPINOk">this video from Tom Scott</a> about invasive Asian carp. What began decades ago as a well-meaning experiment to control algae in US reservoirs and treatment ponds has turned into an ecological disaster. Algae-chomping carp escaped into the wild and did what they do best: chomp some more. They wiped-out lake and river ecosystems, consuming all available resources and crowding out the native life. The video is worth a watch, even if just for the sobering visual of a river so teeming with jumpy carp that <em>boats need overhead netting to protect passengers from getting concussions</em>.</p> <p>Naturally, these carp have me thinking about AI.</p> <p>Maggie Appleton&rsquo;s <a href="https://maggieappleton.com/ai-dark-forest">The Expanding Dark Forest and Generative AI</a> chillingly describes the existential challenge we face on today&rsquo;s Web: <strong>asserting our essential humanity</strong>. As generative AI tools get ever smarter, it will become ever harder to tell if the person you&rsquo;re interacting with online is, in fact, a person at all.</p> <p>Our track record with the <a href="https://en.wikipedia.org/wiki/Turing_test">imitation game</a> so far <a href="https://venturebeat.com/ai/chatgpt-detection-tool-thinks-macbeth-was-generated-by-ai-what-happens-now/">isn&rsquo;t great</a>. If we can&rsquo;t tell artificial from organic, and if those artificial voices speak at a scale and speed we cannot match, then it follows that humans will at some point cease to be the Web&rsquo;s primary users.</p> <p>Rather than a dark forest, the future looks to me a lot more like <strong>an ecosystem that&rsquo;s been taken over by an invasive species.</strong></p> <p>Outnumbered, preyed upon, and starved of resources, humans will flee the open Web for the safety of walled gardens and hidden spaces. This depopulation of the visible Internet is already happening, but AI will dramatically accelerate the trend. Venturing onto the Web will become unwise, and those who do so will need head protection.</p> <p>The best way to handle an invasive species, of course, is to prevent its introduction into a foreign ecosystem in the first place. That ship has sailed, both for AI and the carp.</p> <p>That leaves us with mitigation. There are many ways to deal with invasive species by limiting their spread, containing the damage, and even restoring impacted ecosystems. None of the options are easy or entirely harmless, and success is never guaranteed. But when the stakes are high, the impractical becomes necessary. Just watch the video linked above, which focuses on one rather hardcore solution to the carp problem: <em>electrocuting an entire river</em>.</p> <p>We need to start thinking about mitigation strategies to ensure the Web ecosystem survives its hungry new invader. Ideally, before we find we ourselves turning to such extreme measures.</p> <p><em>(Image source: Stable Diffusion, naturally)</em></p> Storium is coming to Kickstarter in early 2014 https://uniquehazards.com/2013/11/12/hstorium-is-coming.html Tue, 12 Nov 2013 10:33:00 -0700 http://stlhood.micro.blog/2013/11/12/hstorium-is-coming.html <img src="https://stlhood.micro.blog/uploads/2023/728227ce98.jpg" /> <p>From protagonistlabs: &ldquo;We’re happy to share some great news: in the next few months we will be launching a Kickstarter campaign for <a href="https://storium.com">Storium</a>, the online storytelling game.&rdquo;</p> <p>That’s right, we’ve decided to crowdfund the next phase of Storium’s development. Crowdfunding is a perfect fit for our project because it lets us directly validate player interest in Storium while keeping our team focused on building the things our users want.</p> We're working with J.C. Hutchins https://uniquehazards.com/2013/03/26/hpsyched-to-be.html Tue, 26 Mar 2013 06:31:00 -0700 http://stlhood.micro.blog/2013/03/26/hpsyched-to-be.html <img src="https://stlhood.micro.blog/uploads/2023/8173f9de40.jpg" /> <p>If you haven’t read the “7th Son” series of sci-fi thrillers, you really should. They’re about a government human cloning project gone disastrously awry. It’s an epic story chock-full of bizarre technologies, memorable characters, and endless plot twists. I adore these books and ever since I read the first installment I’ve been hoping to someday collaborate with their author, the most excellent <a href="https://jchutchins.net/about-jc/">J.C. Hutchins</a>.</p> <p>That day has finally come: today my startup Protagonist Labs announced that J.C. has joined us as an advisor.</p> <p>In addition to being a skilled and wildly creative writer, J.C. is a respected practitioner in the emerging field of “transmedia”, a discipline that blends different media and narrative techniques to tell stories that involve the audience and span many platforms (including print, web, mobile, TV, film, and games).</p> <p>When I met J.C. he was one of the very earliest people to grok the core ideas behind Protagonist Labs. His enthusiasm and support mean a lot to me and I’m really proud to call him both a friend and, now, a collaborator.</p> Will Hindmarch is our first advisor for Protagonist Labs https://uniquehazards.com/2013/03/13/hwill-hindmarch-is.html Wed, 13 Mar 2013 08:38:00 -0700 http://stlhood.micro.blog/2013/03/13/hwill-hindmarch-is.html <img src="https://stlhood.micro.blog/uploads/2023/5e891588e2.png" alt="" /> <p>Today we announced that <a href="https://wordstudio.net/">Will Hindmarch</a> is our first advisor for Protagonist Labs. I’m over the moon about this news. I mean, I know that’s not physically possible and how would I even breathe in outer space let alone get up there in the first place but shut up I’m excited here okay?</p> <p>Will is a respected designer and writer who has worked across a wide variety of media. While best known for his game design work – where he has been involved in some of the most influential tabletop and video games out there – he is also a writer of great skill and a deep thinker about the creative process.</p> <p>Getting to know Will and inducting him into our circle of trusted allies has been one of the most rewarding experiences for me so far in this new startup. He brings unique perspectives and experience to the table, is a generous collaborator, and is a person of great integrity. He also shares our sense of excitement about the opportunity we’re working on.</p> <p>I’m really proud to know him, and to have him involved in our efforts!</p> Steve Jobs, Google Maps, and the digital afterlife https://uniquehazards.com/2011/11/20/hsteve-jobs-google.html Sun, 20 Nov 2011 04:03:00 -0700 http://stlhood.micro.blog/2011/11/20/hsteve-jobs-google.html <p>Because Google Maps (like other mapping services) utilizes imaging data from multiple sources gathered at different points in time, the same geographic spot can sometimes look very different as you move between zoom levels and views.</p> <p>Weather and seasons can change unexpectedly. Cars can disappear without a trace. Things that were are suddenly no more. It is, quite simply, a time machine.</p> <p>Recently I ran across an example of this effect that is particularly interesting, and bittersweet.</p> <p>For many years Steve Jobs lived in an old Woodside mansion known as the <a href="https://en.wikipedia.org/wiki/Jackling_House">Jackling House</a>. Built 85 years ago in the Spanish Colonial Revival style, many considered it historic. And yet Steve purportedly hated it. He wanted to tear it down and replace it with a more modest, modern home. For years he battled preservationists, and then in February of this year he finally won the fight and demolished the house.</p> <p>In searching Google Maps for the house’s location I discovered something unexpected: it’s still there. Kind of. See for yourself. Here’s a screenshot of the map, since at some point soon the imagery will be updated:</p> <img src="https://stlhood.micro.blog/uploads/2023/98bff3aeaf.jpg" /> <p>There the house stands, dilapidated yet still proud. But turn off the 45° imagery mode, and the house is instantly replaced with an image of the current reality: a barren field.</p> <img src="https://stlhood.micro.blog/uploads/2023/c08109ae22.jpg" /> <p>Steve had the right to do what he did, and I’m sure he had his reasons. But I’m still saddened by the Jackling House’s destruction. It was, to my eye, <a href="http://www.terrastories.com/bearings/the-jackling-house-showdown">unique and beautiful</a>.</p> <p>Now all we have of it is this digital afterlife, an echo of something great. Pondering the sad symbolism here is an exercise I leave to the reader.</p> Remembering how I got here https://uniquehazards.com/2011/10/05/hremembering-how-i.html Wed, 05 Oct 2011 15:43:00 -0700 http://stlhood.micro.blog/2011/10/05/hremembering-how-i.html <img src="https://stlhood.micro.blog/uploads/2023/fb580656d4.jpg" /> <p>The first time I saw a computer – an Apple ][+ in my elementary school’s library.</p> <p>The first time I made the turtle move in Logo.</p> <p>The day the guidance counselor nervously asked my mother, “Do you know how many floppy disks your son has?”</p> <p>The first time they kicked me out of the local library for cheering too loudly while playing Karateka on their Apple //e.</p> <p>The excitement of the day we brought home my first computer, an Apple //c.</p> <p>The time I cut the end off of a phone cable, stuffed the wires into the serial port, wrote a simple program to parrot-back canned dialog, and used the whole setup to trick a friend into thinking I was pulling a real-life “Wargames”.</p> <p>The hours spent exploring local BBSs with my 1200 baud modem. The excitement of downloading illicit software, 140KB at a time.</p> <p>The day my parents bought me an Apple IIGS for Christmas. (Later, my attempts to mask my disappointment at realizing they had neglected to also purchase an Apple 3.5&quot; drive, thus relegating said computer to the role of a slightly-faster Apple //c.)</p> <p>The many years of tinkering, exploration, and happiness that computer ultimately gave me.</p> <p>—-</p> <p>On this sad day my spirits are somewhat lifted by the realization that I can trace much of my life’s trajectory back to these early experiences. I am the person I am today in no small part due to Steve’s work. And I am profoundly grateful.</p> <p>Oh, and that first computer? I’ve still got it. I think that says it all.</p> <p>Now, if you’ll excuse me, I’m going to go fire up Karateka.</p> <p>Steve Jobs: 1955 - 2011</p> Unlit Social Graphs and the Post-Friend Economy https://uniquehazards.com/2011/04/08/hunlit-social-graphs.html Fri, 08 Apr 2011 05:04:00 -0700 http://stlhood.micro.blog/2011/04/08/hunlit-social-graphs.html <img src="https://stlhood.micro.blog/uploads/2023/8b3712b31b.jpg" /> <p>Lawrence Coburn of <a href="https://thenextweb.com/">The Next Web</a> wrote yesterday about “<a href="https://thenextweb.com/news/the-unlit-social-graph">The Unlit Social Graph</a>”. It’s a well-framed article and worth a read. Coburn’s premise is that Facebook, despite its pervasiveness and dominance, does not represent the only type of social graph in our lives. There are many other networks out there but they currently lie dormant. Tremendous opportunity exists for startups that can “light up” these networks: &ldquo;Every school is a network, every employer is a network, every bar is a network, every office building is a network, every hobby is a network, every neighborhood is a network, and at an extreme level, every shared interest is a network, regardless of location.&rdquo;</p> <p><a href="https://battellemedia.com/archives/2011/04/watch_this_space_the_next_generation_of_social_networks_wont_look_like_facebook">John Battelle adds</a>: &ldquo;…we are on the precipice of entirely new ways of thinking about our relationship to others as leveraged over digital platforms, and while Facebook may well be the oxygen or the landmass of this ecosystem, it won’t be the entire ecosystem itself.'</p> <p>Embedded in these statements is a fundamental truth: that not every social connection is equal. Certainly, very few of our social connections truly qualify as “friendship” – at least in the way we formerly used the term. And yet our recent habit has been to paint them all with the same broad stroke.</p> <p>Relationships are shaped by the context in which they exist. Neighborhoods are a great example, wherein the context is a very special location – my home. While we have much to gain from being connected to our neighbors and our neighborhood, we don’t always want to be “friends” with our neighbors. That’s because we can’t easily escape these relationships once they have been created. As a result we now live in a time where most people don’t know their neighbors names. We are living together, yet alone. This is a problem that can’t be solved by a social graph of friends.</p> <p>The good news is that both culture and industry are waking up to the differences between these social graphs and the opportunities they present. I believe that we are entering a new era that I (with tongue in cheek) call the “post-friend economy”.</p> <p>Facebook isn’t going anywhere, of course. But in this world, Facebook’s social graph is one of many we interact with. Instead of seeing every relationship in the context of “friendship”, we use tools that more accurately reflect the diversity of relationships we maintain in the real world.</p> <p>These tools work in a way that makes sense for the context. That means many different levels of personal identity, sharing, and conversation. It means networks created around places, affinity, and time. And perhaps most importantly, it means the ability to unplug and “erase” ourselves from these networks as our own context changes (e.g. when I move to a new neighborhood, when I tire of playing basketball, or when a concert is over).</p> <p>At Blockboard we see ourselves as a post-friend application, one that is focused on the problem of activating neighborhood networks. This is just one of the hundreds of latent graphs that will be activated over the next few years. Things are going to get interesting.</p> We can save del.icio.us, but probably not in the way you think https://uniquehazards.com/2010/12/19/hwe-can-save.html Sun, 19 Dec 2010 05:22:00 -0700 http://stlhood.micro.blog/2010/12/19/hwe-can-save.html <img src="https://stlhood.micro.blog/uploads/2023/0855fb468a.jpg" /> <p>I left Yahoo over two years ago, but prior to that I spent three years running product for del.icio.us.  Since then I’ve remained a loyal user and supporter.  To this day I keep in touch with former del.icio.us colleagues and consider many to be friends.  And though I’ve felt that del.icio.us has been frustratingly slow to evolve in recent years, I’ve always wished the best for the product and the remaining team members.</p> <p>All of this has made last week’s news especially saddening and painful.  First the del.icio.us team was hit by layoffs.  Then it emerged that Yahoo is either <a href="https://techcrunch.com/2010/12/16/is-yahoo-shutting-down-del-icio-us/">shutting down</a> del.icio.us or <a href="https://techcrunch.com/2010/12/17/yahoo-trying-to-sell-del-icio-us-not-to-shut-it-down/">trying to sell it</a>. As I write this it is still not entirely clear what the real story is, but regardless del.icio.us is in peril.</p> <p>An online debate has already begun about various ways that del.icio.us might be “saved”.  As someone who was on the inside for a while and who wants very much to see del.icio.us live on, I thought I’d chime in. For the record, what follows are opinions based on my own experiences. I have not spoken to anyone inside Yahoo about this and I do not have any special knowledge about the current situation.</p> <h2 id="convincing-yahoo-to-keep-investing-in-delicious">Convincing Yahoo to keep investing in del.icio.us</h2> <p>This is unfortunately a non-starter. Last week much of the team was laid off and my guess is that the product is now at best staffed for “maintenance mode”. This sends a fairly clear message and it’s not something Yahoo can easily reverse because they have lost already-scarce expertise in both the product and the complicated technology stack that underpins it.</p> <h2 id="selling-delicious-to-a-third-party">Selling del.icio.us to a third-party</h2> <p>This certainly seems like the best option for del.icio.us and its users, and I hope that Yahoo is able to pull it off. But it’s not a straightforward proposition.</p> <p>As mentioned above, most of the team is now gone. Last week’s leak (and the subsequent fallout) also did unfortunate damage to the del.icio.us brand, sending panicked users to competing products.</p> <p>But ultimately the real challenge here will be the technology. During my time at del.icio.us we rebuilt the entire infrastructure to deeply leverage a number of internal Yahoo technologies. It’s all great stuff but not exactly easy to remove or replace.  Yahoo may have to license some of this technology to the buyer. I’m not sure they’ve done that before.</p> <h2 id="open-sourcing-delicious">Open sourcing del.icio.us</h2> <p>This is a seductive concept but doesn’t make much sense.  As in the case of a sale, they would need to unwind a bunch of proprietary technologies before this could happen.  And open sourcing a complex product isn’t as simple as switching your GitHub repository from private to public.  It involves a lot of work to clean up and document the source.  For del.icio.us this would add up to a huge effort that would be hard to justify purely on a financial basis. Even then, it’s not clear how an open source social bookmarking system would work, given that much of its value comes from being centralized.</p> <h2 id="donating-delicious-to-the-library-of-congress-or-the-smithsonian">Donating del.icio.us to the Library of Congress or the Smithsonian</h2> <p>Now we’re getting closer.  While it is folly to assume either of these institutions could take over del.icio.us and keep it running as a viable service, it does seem like they would be interested in preserving the del.icio.us corpus and making it available for research.</p> <p>I love del.icio.us for many reasons, but chief among them is that it is the Internet’s memory storage device.  In the 7+ years of its existence it has recorded the collective online journeys of millions of users during a time when the Web was evolving dramatically.  Those memories are irreplaceable and have enormous value both to their owners (the users) and to society.</p> <p>And so this is where we end up: del.icio.us may or may not have a future as a service, but regardless we can still “save it” by extracting and preserving its collective memories.  There are two ways to do this:</p> <ul> <li>Yahoo could proactively release the corpus of publicly-shared bookmarks and tags. This could take the form of a mass data dump into the public domain, or it could be via an agreement with an institutional partner (much like <a href="https://blogs.loc.gov/loc/2010/04/how-tweet-it-is-library-acquires-entire-twitter-archive/">Twitter did earlier this year</a>).</li> <li>The del.icio.us user community could organize to save the data themselves via a coordinated harvesting project.</li> </ul> <p>The second approach could produce valuable results but would require no shortage of cleverness in order to avoid triggering rate limiters and other abuse mitigation mechanisms.  Even then it’s not clear that the entire corpus is currently accessible in this manner. The first approach would be much more direct and complete, and would likely earn back for Yahoo some of the goodwill it has recently lost.</p> <p>Separately from the public data, there is the issue of personal user data. While del.icio.us has long had a bookmark export tool, other pieces of personal or private data are not easily exportable, notably the user’s Inbox (links shared with them by other users) and their Network (links saved by users they follow, as well as the list of those contacts – it’s basically Twitter for bookmarks). The user community has already started working on this problem: former del.icio.us engineering lead Josh Whiting has written a Ruby script that exports your Inbox. Ideally Yahoo should provide official tools for exporting this data.</p> <p>In conclusion, releasing the public corpus is the right thing to do for del.icio.us, for Yahoo, and for the Internet. If a sale proves difficult – or even if it succeeds – I hope Yahoo will take this path and I would strongly encourage them to do so.</p> Google Instant is an example of how Yahoo could have won in search https://uniquehazards.com/2010/09/08/hgoogle-instant-is.html Wed, 08 Sep 2010 08:16:00 -0700 http://stlhood.micro.blog/2010/09/08/hgoogle-instant-is.html <img src="https://stlhood.micro.blog/uploads/2023/543d9c5721.jpg" /> <p>Today <a href="https://techcrunch.com/2010/09/08/google-instant-its-search-before-you-type/">Google launched Google Instant</a>, and make no mistake: this is big. It’s far more than just a new fancy interface. It’s a fundamental change to a user interaction model that’s been largely unchallenged for years. It also represents a significant financial and technical commitment on Google’s part (all those new searches and suggestions aren’t free). Last but not least, it clearly demonstrates that Google still has both the means and motivation to challenge a status quo they themselves helped create.</p> <p>By all accounts this is a bold and brave innovation. Which is why it may surprise you to learn that Google Instant is actually five years old. Yahoo built it back in 2005.</p> <p>Yes, Yahoo. In 2005.</p> <p>I can see you’re confused. I don’t blame you. Let’s back up.</p> <p>Back in 2005, prior to joining the del.icio.us team, I was a product manager on Yahoo’s search team. I led the (truly awesome) team that built and shipped Yahoo’s first AJAX search tools. Our biggest project and crowning achievement was LiveSearch: a fully interactive search UI that produced auto-complete search suggestions and displayed instant search results – in real-time as you typed.</p> <p>It was slick. It was extremely useful. It was a potential game-changer. And it was almost exactly the same product as Google Instant.</p> <p>There were differences of course. The search suggestions were displayed on the left side of the results, instead of above them like in Google’s implementation. This was to avoid the distraction of shifting the result set down the screen as the user typed. The relevance of Yahoo’s results at the time were likely lower than what Google is capable of displaying today, but they were still quite good. And the layout clearly needed further optimization, since the default Yahoo SERP at the time was top-heavy with ads and in-network content “shortcuts” (some of which were quite useful, others not so much).</p> <p>All in all, my assessment is that Google Instant is a superior implementation, but LiveSearch was close. Damn close. It’s something Yahoo should be proud of, and it’s a shame more people don’t know about it.</p> <p>There is basically no trace left of LiveSearch these days, but <a href="http://blogoscoped.com/archive/2006-05-10-n17.html">here’s an article about it from Google Blogoscoped</a>. And below is a screenshot, taken from the same article:</p> <img src="https://stlhood.micro.blog/uploads/2023/55b3969d41.png" /> <p>So what the frak happened? If this innovative product was built nearly 5 years ago (it actually shipped in early 2006), why hasn’t anyone heard of it?</p> <p>The answer is (mostly) in the screenshot. Yahoo would not let us ship LiveSearch on yahoo.com or as a part of Yahoo’s search engine. Instead we were only allowed to launch it on AllTheWeb, a smaller, lower-traffic search engine that Yahoo had acquired years earlier and largely left to atrophy. (In comparison, Google just launched it on google.com. Boom.)</p> <p>You have to remember that, at the time, Yahoo’s search business was doing just well enough that there was very little institutional appetite for product risk. As a result, “big” or disruptive ideas were too often left to whither on the vine. By focusing on the <a href="https://52weeksofux.com/post/694598769/the-local-maximum">local maximum</a>, Yahoo unwittingly traded innovation for incremental optimization.</p> <p>LiveSearch was thus relegated to a tiny test bucket of users who didn’t actually use Yahoo’s search product (or any modern search engine). Usage data from this flawed test was used to internally evaluate its success in comparison to the model it was actually trying to disrupt. Lacking high-level support for its larger vision and starved for resources, LiveSearch was understandably put out to pasture.</p> <p>To be fair, this all happened several years ago and user attitudes have changed since then. An interaction paradigm that today seems brilliant was back then perhaps not as immediately accessible to users. I also look back and feel that I could have more effectively “sold” the vision to Yahoo’s management and fought for it. Ah, the benefits of hindsight.</p> <p>But still, here we are, five years later. And what is essentially the same product is launching to most of the world’s online audience in one fell swoop. Clearly an opportunity was lost somewhere along the way.</p> <p>While Yahoo’s reputation as an innovator has suffered in recent years, I can tell you first-hand that there was no shortage of amazing ideas inside those walls, some of them way cooler and more disruptive than LiveSearch. Nor was there any shortage of smart, driven, and creative people on the payroll. I worked with a bunch of them.</p> <p>Instead, what was lacking was the ability to bust past that local maximum. Google may have challenges of its own these days, but today it clearly demonstrated that it still has the organizational courage to challenge its own preconceptions. As I read the coverage and play with Google Instant, I can’t help but wonder how things might have gone if Yahoo had shared this trait five years ago. It could have been pretty damn great.</p> Initiating Stage 2 https://uniquehazards.com/2010/05/27/hinitiating-stage-h.html Thu, 27 May 2010 02:49:00 -0700 http://stlhood.micro.blog/2010/05/27/hinitiating-stage-h.html <img src="https://stlhood.micro.blog/uploads/2023/924a5c3e08.jpg" /> <p>So the news is out: we’ve raised our first round of funding for BlockChalk.</p> <p>It turns out that today is an especially appropriate day for the news to break. I’m currently in Washington D.C. for O&rsquo;Reilly’s Gov 2.0 Expo, the leading conference on open government and citizen engagement. These values are at the heart of what we’re doing with BlockChalk. It feels pretty great to be here with so many people who feel similarly.</p> <p>We put a ton of effort into the fundraising process and I couldn’t be more proud of the roster of investors who have stepped up to support us. They represent a broad range of backgrounds, perspectives, and geographies. That matches well with our goals for BlockChalk. We plan to grow <a href="https://uniquehazards.com/2010/02/17/hliving-in-the.html">beyond the bubble</a> and reach people all over the country and all around the world.</p> <p>This is definitely the most exciting moment in my professional life. It’s been a lot of work getting to this point, and I’m psyched to have reached it (note: understatement of the year). But this is just the first marker on a long road ahead.</p> <p>Now the real work starts. And the real fun.</p> My talk from Where 2.0 https://uniquehazards.com/2010/04/01/hmy-talk-from.html Thu, 01 Apr 2010 14:50:00 -0700 http://stlhood.micro.blog/2010/04/01/hmy-talk-from.html <p><a href="https://www.slideshare.net/stlhood/bringing-geo-home-to-roost-3618108"><img src="https://stlhood.micro.blog/uploads/2023/e5e428ba8c.png" width="600" height="447" alt="" /></a></p> <p>O&rsquo;Reilly’s <a href="https://where2conf.com/where2010/">Where 2.0</a> conference has just wrapped up. I’ve been to my share of conferences lately but Brady Forrest and company really outdid themselves with this one. The entire experience was solid at every level of detail, from the big (interesting and relevant keynotes and sessions) to the small (food and drinks that actually didn’t suck).</p> <p>I had the pleasure of giving a talk at Where 2.0 this year, entitled “<a href="https://www.slideshare.net/stlhood/bringing-geo-home-to-roost-3618108">Bringing Geo Home To Roost</a>”. My presentation explored various ways that we might take geolocal services beyond their early-adopter status and make them broadly appealing to the mainstream masses. Fortunately, BlockChalk serves as an interesting reference here.</p> SXSW: it's not a war, it's a scouting mission https://uniquehazards.com/2010/03/13/hsxsw-its-not.html Sat, 13 Mar 2010 14:06:00 -0700 http://stlhood.micro.blog/2010/03/13/hsxsw-its-not.html <img src="https://stlhood.micro.blog/uploads/2023/91ef4565bf.jpg" /> <p>As everyone knows, the focus at <a href="https://www.sxsw.com/">South By Southwest</a> this year (at least from a tech perspective) is on location-based services (I’m there representing BlockChalk).</p> <p>Oh, and apparently we’re all at war! Foursquare and Gowalla are at each other’s throats, and the other startups are fighting over the scraps. It’s a Highlander-esque battle royale, and only one fighter can emerge with its head attached to its shoulders and thus claim The Prize. Or at least that’s the story everyone is writing about.</p> <p>That story is inaccurate and it overlooks the real value of this event.</p> <p>Yes, it’s true that Foursquare and Gowalla are in a fight, but what would you expect? They are the biggest names and their products are essentially identical. Look beyond these two players and you’ll find a group of startups that is as diverse as it is large.</p> <p>Entrepreneurs and journalists who think of SXSW as a battlefield are making two key mistakes.</p> <p>The first lies in the implication that “location-based” startups are all competitive with each other. That’s simply not true. Location is not a strategy or a product. It is a feature. BlockChalk is no more competitive with something like Gowalla than lettuce is competitive with asparagus. Just like everyone benefits from eating more vegetables, all location-based startups benefit from increased experimentation with and adoption of geo-local services.</p> <p>The other mistake is in coming to SXSW looking for winners and losers. This is not a contest; it is a chance to talk to real people who use (or don’t use) your product. This is why we talk to users – to learn. Let’s not lose sight of that.</p> <p>SXSW represents a golden opportunity for learning. Don’t miss it because you’re busy sharpening your sword.</p> Don't think you care about privacy? You will... eventually https://uniquehazards.com/2010/02/22/hdont-think-you.html Mon, 22 Feb 2010 04:00:00 -0700 http://stlhood.micro.blog/2010/02/22/hdont-think-you.html <img src="https://stlhood.micro.blog/uploads/2023/fc57070014.jpg" /> <p>In the tech industry it has almost become embarrassing to voice concerns about erosion of personal privacy. After all, <a href="https://www.theguardian.com/technology/2010/jan/11/facebook-privacy">it’s no longer a social norm</a>, right?</p> <p>In this brave new world that Facebook, Twitter, and Foursquare hath wrought, privacy is often seen as an outdated concern. People who speak out to the contrary are branded as Luddites or dinosaurs because, apparently, the future is already written and is already here.</p> <p>But a funny thing happened on the way to the overmind. Google launched a little thing called “Buzz”.</p> <p>Suddenly, privacy is alive again. I can’t even keep up with all the <a href="https://techcrunch.com/2010/02/17/google-buzz-warning-force-feeding-users-can-result-in-vomiting/">articles</a> <a href="https://www.nytimes.com/2010/02/18/technology/personaltech/18pogue.html?partner=rss&amp;emc=rss">bemoaning</a> this latest violation of our souls. The backlash has been quick and it has been brutal. Google has moved quickly and impressively to correct, but the damage is done. And in recent days the outcry has moved beyond Buzz to target location-based services like Latitude, Foursquare and Gowalla.</p> <p>So which is it? Is privacy pointless and dead or alive and kicking? What’s going on here?</p> <p>In a nutshell, we’re seeing that everyone cares about privacy…eventually.My hypothesis is that everyone has their own “privacy threshold” which is directly tied to how much they have to lose and how obviously those things are threatened. With Buzz, Google created a situation that triggered both of those alarms at the same time, for a huge number of people and at a very low threshold.</p> <p>Don’t think you have much to lose? Don’t feel like it’s threatened yet? Just wait a little longer.</p> <p>When we’re younger, we simply don’t have a lot of history. Our tracks in the snow are few and our impact on the world has been limited (unless you’re Mozart or Wesley Crusher). In short, we either don’t have a lot of “life data&quot; or we haven’t yet realized that we value it. We therefore have less to lose if that data is used in ways we weren’t expecting.</p> <p>Things change as time goes by. The more stuff we do (e.g. the more we “live”), the more life data we accrue. We build history, relationships, possessions, opinions, accomplishments and failures. We each accrue different life data at different rates, so it’s more about our personal experiences than it is about our physical age. The more life data we’ve accrued, the more we have to lose when that data is misused.</p> <p>Many people claim that privacy is a generational concern that will soon be irrelevant. And I certainly concede that the “social norms” have changed. But my point is that privacy is not generational but rather is driven by life experience. Even today’s generation will start caring about privacy at some point.</p> <p>When Google launched Buzz as an integrated part of Gmail, they linked it to what is for many people — regardless of life experience — a significant repository of accrued life data, and one that is assumed to be private. Email brings with it certain assumptions, but Buzz violated those in <a href="https://avc.com/2010/02/explicit-vs-implicit-social-nets/">unexpected</a> but immediately obvious ways.</p> <p>As a result the Buzz launch became one of the most easily recognizable and least expected erosions of online privacy to date. It’s no wonder the backlash has been so severe: Buzz showed a huge number of people just how much they have to lose and clearly illustrated the immediacy of the threat.</p> <p>But it’s not all bad news.</p> <p>Personally I think this fiasco has done a tremendous favor for both the public and the industry. It has woken up a new set of users to the consequences of privacy erosion. And it has put some of today’s hottest tech companies on notice that this is something they need to get smarter about. I think we’ll ultimately see better, safer products and wider adoption as a result.</p> <p>And that’s something to buzz about.</p> Living in the bubble (or, why the future of location is even bigger than you think) https://uniquehazards.com/2010/02/17/hliving-in-the.html Wed, 17 Feb 2010 02:54:00 -0700 http://stlhood.micro.blog/2010/02/17/hliving-in-the.html <img src="https://stlhood.micro.blog/uploads/2023/7ce0da294e.jpg" width="600" height="433" alt="" /> <p>Last Friday I attended TechCrunch’s excellent Real Time CrunchUp in San Francisco. Real time web services are all the rage these days of course, and this conference brought together entrepreneurs, engineers, investors, and others to discuss the field and debate where it’s going.

</p> <p>Much of the focus was on location-based services and information “streams”. Since this is the area in which BlockChalk plays, the discussion was of personal interest to me. Companies like Twitter and Foursquare were in the spotlight, although newcomers like SimpleGeo, GeoAPI, and HotPotato attracted their share of well-deserved attention. Great products, smart people.

</p> <p>As I listened I heard some exciting predictions for the future: how one day soon we would all know where everyone is all the time; that people everywhere would share such information willingly and benefit from it greatly; and how this would fundamentally alter the way we interact as a society.

</p> <p>But something about all this didn’t feel quite right. There seemed to be an underlying assumption at play: that today’s location-based services show us the shape of things to come. And so it was that about halfway through the day I finally realized what was bothering me.</p> <p>Everyone in the room was living in the geo bubble.
</p> <p>What’s the geo bubble? It’s a land populated by the early adopters of today’s location-based services. Inside the bubble, people’s online actions are primarily driven by social activity and personal reputation. This has many implications, but the one I want to address here is privacy: bubble-dwellers have a reduced expectation of it, because it gets in the way of the things they want to do.</p> <p>

Now, the bubble is a great place to live, and bubble-dwellers are perfectly nice folks. It’s not my intent to besmirch them (hell, I frequently visit the bubble myself). Instead, my intent is to point out that, by definition, there is a world outside the bubble. That’s where most people live, and yet as an industry we’ve barely scratched the surface of what can be done there.

</p> <p>Today’s hottest services are pitched directly at bubble-dwellers, and by all accounts they are popular, useful, and fun. But by linking your identity to your location and sharing this information broadly, many of these services largely ignore issues of personal privacy and security. As a result, there are a wide range of everyday social interactions and transactions to which they are ill suited — buying and selling goods and services, lodging anonymous complaints, reporting crimes, the list goes on. It also means that a large portion of the population may never feel comfortable using them.</p> <p>

Dave and I created BlockChalk in part because we believe that the world outside the bubble is every bit as interesting as (and larger than) the world inside. Bubble-based apps will undoubtedly continue to grow dramatically, and the bubble itself will grow as early adopter behavior trickles down to a broader audience. But in order for location-based services to truly reach the mainstream, we as product designers will need to get even smarter about the social assumptions that we are harnessing — or in some cases, undermining.</p> <p>

For BlockChalk, that means a focus on personal privacy. We’re building it from the ground up to be a location-based service for everyone, where the user is always in control of how much identity and location information they share. We’ve also made it aggressively hyper-local, with a strong focus on what’s going on in your neighborhood. We think this will encourage people to use BlockChalk for completely different purposes than systems like Twitter and Foursquare. We also think it will attract entirely new types of users to this space. And we’re already seeing both happening.</p> <p>

The world of location-based services is moving faster than ever, and the hottest products out there today are innovative and fun to use. But it would be a mistake for us to assume that today’s users are representative of the overall population, and that today’s products necessarily represent the shape of things to come.

</p> <p>The future is going to be different — and even bigger — than we expect.</p>