Hogwash

FirePig's blog on web development, usability, data collection, and anything shiny that happens to catch our eye.

Improving Online Data Collection

Wednesday, October 07, 2009 at 6:41 AM by Andrew

We here at FirePig have recently begun work on a new product: an online data collection and reporting tool for large scale collection efforts. About ten years ago (or, in Internet-speak, "back in the Paleolithic Era") I built such a system for my employer at the time. It had a fair number of neat features, and we'll be drawing on that experience as we move forward. However, the web has come a long way in the past decade, and we're convinced that we can use modern techniques to significantly improve the state of electronic data collection. We're not talking about simple polls or surveys with a handful of questions. While you could certainly use what we're building for those sorts of projects, there are plenty of existing tools that can handle that. We're talking more involved, complicated data collections - imagine a tool that could handle the Census long form, and you have a sense of what we're aiming for.

We're in the initial planning stages right now, and we'd like your help. Please share with us in the comment section your experiences with providing information on the web: the good, the bad and the ugly. We want to hear from everyone, no matter how web savvy you are. (In fact, the less savvy, the more useful your experiences will be.) If you have any suggestions for features, we'd love to hear those, too.

Bookmark and Share

User Comments

4 comment(s) for “Improving Online Data Collection”

  1. Gravatar of Steve Bell
    Steve Bell Says:
    There's a giant blank space at the top of this page, before the Hogwash heading (in Firefox 3.5.3). My 2 cents:
    1. Have all fields/questions on one page.
    2. It's nice when all fields are required (don't want to waste time looking for the asterisks). Who wants to give out more info than is necessary?
    3. It's a pain to wade through cookies deciding which ones to save when cleaning the cache, so in addition to checking "Remember Me" boxes it'd be nice to have a link to the browser's cookie option window, where an option to save a site's cookie *after* cleaning the cache is given. Like a permanent cookie.
    4. Obverse to above: give a third option to not save *any* info, on the remote or local machine, regardless of cookie settings.
    5. Give a nice big window in which to make comments.
    Wednesday, October 07, 2009
  2. Gravatar of Dave Broudy
    Dave Broudy Says:
    Are you doing anything to support skip lists? I think that's something that becomes more important as the size of a survey grows. A UI on the frontend that condenses not applicable questions would be pretty simple, but even better would be a tool for the survey creator on the backend.
    Wednesday, October 07, 2009
  3. Gravatar of Andrew @ FirePig
    Andrew @ FirePig Says:
    @Steve: I'm in 3.5.3, but don't see the space you mention. Let me know via email if it's still happening.

    You bring up a lot of good issues. I'll try to tackle them in order:

    1.For the types of data collections we're talking about, having everything on a singe page isn't always feasible - I've seen surveys with over 500 individual questions! I think the real issue here is they ability to navigate easily through the different pages, including a way to get back to old responses, and a way to bookmark your place if you need to come back later.

    2. One of the big questions is determining exactly what "required" means. In a big survey, if you stop someone in their tracks for not answering a particularly tough question, they may well leave and never come back. At the same time, some questions really are optional, or less important, and providing a user with clues as to what is most important to you can be extremely beneficial.

    3. Not sure we can do that on a web page, given the wide variety of browsers in question. If it is, I'd be a little bit afraid to encourage those with little computer experience to play with their settings.

    4. A cookieless option is a possibility.

    5. Heh. Point taken!
    Wednesday, October 07, 2009
  4. Gravatar of Kearns
    Kearns Says:
    I really like it when my answers make entire pages of questions disappear, I don't like having to scroll past stuff that isn't applicable to me.

    I do like a status bar showing me how far I have to go.

    I do like it when the tab order is correct.

    I do like forms that don't require me to ever touch the mouse.

    I don't like "hidden requirements" that only tell me about something (or even worse tell me something incorrect) when I try to submit the form.

    I don't like it when the form can't be submitted, but I have to go searching for why.

    I don't like form errors to be displayed in red as if everyone in the world can see red.

    I don't like ajax that breaks up the flow of the page. If there are 3 levels of drop down, I shouldn't have weird flickering as my selections in one loads the next.

    I do like data gathering systems that once I realize it is a ton more work than I thought, I can save my progress and come back later.

    I'm sure there's more, but that's a good start, eh?
    Wednesday, October 07, 2009

© 2009 FirePig Partners LLC. All rights reserved.