Skip to content

My thoughts about Fargo.io using fargo.io

Organizing Your Life With Python: a submission for PyCon 2015?

I have penciled into my calendar a trip  to Montreal to attend PyCon 2014.   In my moments of suboptimal planning, I wrote an overly ambitious abstract for a talk or poster session I was planning to submit.  As I sat down this morning to meet the deadline for submitting a proposal for a poster session (Nov 1), I once again encountered the ominous (but for me, definitive) admonition:

Avoid presenting a proposal for code that is far from completion. The program committee is very skeptical of "conference-driven development".

It's true: my efforts to organize my life with Python are in the early stages. I hope that I'll be able to write something like the following for PyCon 2015.

Organizing Your Life with Python

David Allen's Getting Things Done (GTD) system is a popular system for personal productivity. Although GTD can be implemented without any computer technology, I have pursued two different digital implementations, including my current implementation using Evernote, the popular note-taking program. This talk explores using Python in conjunction with the Evernote API to implement GTD on top of Evernote. I have found that a major practical hinderance for using GTD is that it way too easy to commit to too many projects. I will discuss how to combine Evernote, Python, GTD with concepts from Personal Kanban to solve this problem.

Addendum: Whoops…I find it embarrassing that I already quoted my abstract in a previous blog post in September that I had forgotten about. Oh well. Where's my fully functioning organization system when I need it!

Tagged ,

Current Status of Data Unbound LLC in Pennsylvania

I'm currently in the process of closing down Data Unbound LLC in Pennsylvania.  I submitted the paperwork to dissolve the legal entity in April 2013 and have been amazed to learn that it may take up to a year to get the final approval done.  In the meantime, as I establishing a similar California legal entity, I will certainly continue to write on this blog about APIs, mashups, and open data.

Must Get Cracking on Organizing Your Life with Python

Talk and tutorial proposals for PyCon 2014 are due tomorrow (9/15) .  I was considering submitting a proposal until I took the heart the appropriate admonition against "conference-driven" development of the program committee.   I will nonetheless use the Oct 15 and Nov 1 deadlines for lightning talks and proposals respectively to judge whether to submit a refinement of the following proposal idea:

Organizing Your Life with Python

David Allen's Getting Things Done (GTD) system is a popular system for personal productivity.  Although GTD can be implemented without any computer technology, I have pursued two different digital implementations, including my current implementation using Evernote, the popular note-taking program.  This talk explores using Python in conjunction with the Evernote API to implement GTD on top of Evernote. I have found that a major practical hinderance for using GTD is that it way too easy to commit to too many projects.  I will discuss how to combine Evernote, Python, GTD with concepts from Personal Kanban to solve this problem.

 

Embedding Github gists in WordPress

As I gear up I to write more about programming, I have installed the Embed GitHub Gist plugin. So by writing

[gist id=5625043]

in the text of this post, I can embed https://gist.github.com/rdhyee/5625043 into the post to get:

Tagged ,

Working with Open Data

I'm very excited to be teaching a new course Working with Open Data at the UC Berkeley School of Information in the Spring 2013 semester:

Open data — data that is free for use, reuse, and redistribution — is an intellectual treasure-trove that has given rise to many unexpected and often fruitful applications. In this course, students will 1) learn how to access, visualize, clean, interpret, and share data, especially open data, using Python, Python-based libraries, and supplementary computational frameworks and 2) understand the theoretical underpinnings of open data and their connections to implementations in the physical and life sciences, government, social sciences, and journalism.

 

A mundane task: updating a config file to retain old settings

I want to have a hand in creating an excellent personal information manager (PIM) that can be a worthy successor to Ecco Pro. So far, running EccoExt (a clever and expansive hack of Ecco Pro) has been a eminently practical solution.   You can download the most recent version of this actively developed extension from the files section of the ecco_pro Yahoo! group.   I would do so regularly but one of the painful problems with unpacking (using unrar) the new files is that there wasn't an updater that would retain the configuration options of the existing setup.  So a mundane but happy-making programming task of this afternoon was to write a Python script to do exact that function, making use of the builtin ConfigParser library.
"""
compare eccoext.ini files

My goal is to edit the new file so that any overlapping values take on the current value

"""
current_file_path = "/private/tmp/14868/C/Program Files/ECCO/eccoext.ini"
new_file_path = "/private/tmp/14868/C/utils/eccoext.ini"
updated_file = "/private/tmp/14868/C/utils/updated_eccoext.ini"

# extract the key value pairs in both files to compare  the two

# http://docs.python.org/library/configparser.html
import ConfigParser

def extract_values(fname):
    # generate a parsed configuration object, set of (section, options)
    config = ConfigParser.SafeConfigParser()
    options_set = set()

    config.read(fname)
    sections = config.sections()
    for section in sections:
        options = config.options(section)
        for option in options:
            #value = config.get(section,option)
            options_set.add((section,option))

    return (config, options_set)

# process current file and new file

(current_config, current_options) = extract_values(current_file_path)
(new_config, new_options) = extract_values(new_file_path)

# what are the overlapping options
overlapping_options = current_options & new_options

# figure out which of the overlapping options are the values different

for (section,option) in overlapping_options:
    current_value = current_config.get(section,option)
    new_value = new_config.get(section,option)
    if current_value != new_value:
        print section, option, current_value, new_value
        new_config.set(section,option,current_value)

# write the updated config file

with open(updated_file, 'wb') as configfile:
    new_config.write(configfile)

MITH API workshop

I'm excited about the upcoming MITH API Workshop to be held in two weeks from Feb 25-26 at UMD :

The Maryland Institute for Technology in the Humanities will host a two-day workshop on developing APIs (Application Programming Interfaces) for the digital humanities. The workshop will gather 40-50 digital humanities scholars and developers, who along with industry leaders will demonstrate their APIs during this “working weekend.” We will discuss ways that existing and future APIs could be leveraged for digital humanities projects.

As someone who has been fascinated by APIs for years, I hope to learn a lot from my fellow digital humanities about what they care about. One of my tasks is to give an introductory talk about APIs.  What do I want to cover?  I'm still working out the exact structure, but the following topics come to mind:

  • What are APIs.  The relationship between web APIs (the focus of our workshop, I believe) and other APIs
  • How to learn more about APIs
  • APIs of specific interest to the digital humanities, with specific references to Freebase, Google geo-APIs, and OpenLibrary (organizations represented by fellow presenters)
  • Why does REST matter. (I'll only anticipate what fellow speaker Peter Keane will be bring up in his talk about REST)
  • How to consume APIs; What are mashups
  • How to deploy APIs
  • Open questions I think about

Stay tuned. Over the next two weeks, I'll work through these topics for myself (writing on this blog). I'll take this time as an opportunity to revisit what I wrote  in Pro Web 2.0 Mashups:  Remixing Data and Web Services and what I taught in my Mixing and Remixing Information course I taught at UC Berkeley over five years.

Slides for my talk on open government + Freebase

I'm looking forward to giving a little talk on open government + Freebase + Recovery Act data tonight at the Freebase meeting.   I'm deeply excited about the potential of open government data to change how we work with government (not to mention how we understand its workings).    Here are some slides that will help frame my talk tonight.

Tagged ,

Announcing Data Unbound: a new training and consulting firm

In 2006, after weblogging for some 6 years while working at UC Berkeley, I took on a new role as a data architect on campus.  I felt it important to keep blogging about my professional interests but to do so under a new moniker.  I came up with "data unbound" to name the passion I had for the myriad possibilities  latent in data, some of which I have strived to reveal.

A lot has happen since I started dataunbound.com, the weblog.  I left my staff position at UC Berkeley so that I could devote myself more fully to the task of teaching others about the world of web APIs and mashups.   I wrote my book on the subject Pro Web 2.0 Mashups: Remixing Data and Web Services, which has been very well-received, I'm pleased to say.  Right now, I'm teaching my course Mixing and Remixing Information for the fifth time at the School of Information at UC Berkeley.  This year, I'm focusing the course on the rapidly expanding area of open government and the web.

And now, I (in partnership with my wife, Laura Shefler) have taken the next step of formally starting Data Unbound LLC:

Data Unbound LLC is a training and consulting company that helps organizations access and share data effectively. The value of your data, when it is scattered throughout multiple databases and applications, grows if you can make it all work together. This value increases further when you leverage your information resources with the vast world of data on the Web. Our specialty is helping you to use APIs (application programming interfaces) to integrate data across your organization and beyond.

We're open for business, ready  to work with clients  to solve their data problems.  Our training will enable their organizations to integrate data, both their own and that of others through APIs and data standards.  I encourage you to read more of what we have written on dataunbound.com, in which we detail our approach and our offerings.  In the next months, I'll be describing how general principles behind data integration and web APIs can solve your  problems in your specific context. And if you know anyone who make use of Data Unbound, by all means, put them in touch with us.