Kin Lane

My thoughts on technology and life

Scalr

What is Scalr?

Scalr is software that creates redundant, self-curing, and auto-scaling clusters to run your website. Using EC2 and Cloud Computing.

It allows you to create server farms through a web-based interface using prebuilt AMI’s for load balancers (pound, nginx, or Amazon’s load balancing service), app servers (apache, rails, others), databases (mysql master-slave, others), and a generic AMI to build on top of.

The health of the farm is continuously monitored and maintained. When the Load Average on a type of node goes above a configurable threshold a new node is inserted into the farm to spread the load and the cluster is reconfigured. When a node crashes a new machine of that type is inserted into the farm to replace it.

Multiple AMI’s are provided for load balancers, mysql databases, application servers, and a generic base image to customize. Scalr allows you to further customize each image, bundle the image and use that for future nodes that are inserted into the farm. You can make changes to one machine and use that for a specific type of node. New machines of this type will be brought online to meet current levels and the old machines are terminated one by one.

The project is still young, but we’re hoping that by open sourcing it the AWS development community can turn this into a robust hosting platform and give users an alternative to the current fee based services available.

Full Conectivity in the Cloud

Full connectivity: Government policies should encourage the deployment of wired and wireless broadband access so that users can access cloud-based services anytime, anywhere.

Living-Stories – A New Format for Online News

Living Stories are a new format for presenting and consuming online news. The basic idea of a living story is to combine all of the news coverage on a running story on a single page. Every day, instead of writing a new article on the story that sits at a new URL and contains some new developments and some old background, a living story resides at a permanent URL, that is updated regularly with new developments. This makes it easier for readers to get the latest updates on the stories that interest them, as well as to review deeper background materials that are relevant for a story’s context.

To see Living Stories in action, go to http://livingstories.googlelabs.com and click on one of the listed stories. These stories were created for a Labs experiment that ran from Dec 2009 – Feb 2010 by a partnership between Google, the New York Times, and the Washington Post. This open source package lets you create similar living stories on your own website.

Read more in the blog post announcing the open source launch of Living Stories.

Grouping and targeting my Google Buzz

I was spending time organizing my Google Contacts.  Up until now they really didn’t “require” any sort of organization.

Then I was playing with Google Buzz and seeing how I can use as part of my communication and production platform.

With an organized Google Contacts and logical groups,  I could get very fine tuned in my Google Buzz targeting.

They allow you to to quickly mark your buzz as private, then distribute to certain groups:

One thing I noticed though is very few people in my groups actually use Buzz.  So how does this apply?

New Billing Structure for Amazon Web Services (AWS)

Amazon Web Services has added a new and more flexible way to manage an account.  They have added the ability for:

• Consolidated billing across multiple AWS accounts
• Volume pricing across more than one AWS account
• Way to track the cost of a project across an organization.

The new billing feature lets you designate one AWS account as a paying account and a set of other accounts as linked accounts to form a simple one-level hierarchy. The AWS usage within the linked accounts is rolled up into the paying account for volume pricing and billing purposes, so there’s just one AWS bill per month.

The account relationship is solely for accounting purposes and normal AWS permissions still apply. You can set up AWS accounts for projects, departments, dev/test/staging/production, or even by employee. You can add up to twenty linked accounts to a single paying account.

These features will do a lot to address the needs of larger customers.. Later this year  Amazon Web Services is planning to provide additional functionality to allow linking of accounts and user permissions.

Building Network Around Common Interests

I have explored the sharing and social networking options within my Google Reader.  My network has not really grown or evolved.  Ocasionally I found someone who had liked an article I really liked and I would add them to my Google Reader network.

Since I started using Google Buzz it has brought this social network out in front.  It is forcing me to clean up my Google Contacts and starting grouping my network.

I spend my entire day in Google Reader.  It is logical to build a business and professional network here and to establish and build relationships based upon common interests.

It is very interesting how the Google Platform is really coming together.

Google clears up a few things on Google Buzz

Google responded to everyone on the Gmail blog about some of the open questions around Google Buzz.  They addressed some of the privacy concerns around your Google Contacts and how your Buzz network operates.

They cover:

• More visible option to not show followers/people you follow on your public profile
• Ability to block anyone who starts following you
• More clarity on which of your followers/people you follow can appear on your public profile

I like how they closed it:

We designed Buzz to make it easy to connect with others and have conversations about things that interest you, and it’s great to see millions of you doing this already. It’s still early, and we have a long list of improvements on the way. We look forward to hearing more suggestions and will continue to improve the Buzz experience with user transparency and control top of mind.

They say they are going to “improve the Buzz experience with user transparency and control top of mind”.   Have to think about that one.  Transparency and privacy with Google Buzz will an interesting line to draw.

Google Buzz API

Over the next several months Google Buzz begin rolling out APIs for developers. Our plans involve full/read write support for updates with the Atom Publishing Protocol, rich activity notification with Activity Streams, delegated authorization with OAuth, federated comments and activities with Salmon, distributed profile and contact information with WebFinger, and much, much more. We invite developers to join us in these conversations on the Buzz API Group.

Download my Credit Card Statements

It just drives me nuts that I can’t get good data access to my credit cards.  I just want an easy download of ALL my credit card transactions.

You’d think you could but Chase now B of A.  You just can’t easily.

Ok…I was able to download an easy spreadsheet for last year at American Express.

Top Social Network Promotion on Web Site

I was setting up a demonstration event web site to show how events can become more social.  I setup new accounts for the event at Facebook, Twitter, and LinkedIn.

Facebook has their Fanbox tool that easily allows you to build a widget or badge to showcase your Facebook page on your web site.

Twitter has their profile widget that quickly puts your Twitter stream on your web site.

LinkedIn just has their promote your profile buttons.

Facebook and Twitter gives you ready to go tools for building an advanced widget that extends your profile beyond their network and onto your web site.  LinkedIn just gives you buttons?  They need to step up with some better tools.

UPDATE: Ok…guess I didn’t look hard enough.  They have a selection of widgets at - http://developer.linkedin.com/community/widgets

Personal Budgets

Just came across a cool web application called Bundle that lets you  see how people like you save and spend their money?

You can use Bundle to compare yourself by age, location, income and household status.

I love the tools out there that help us understand our money and spending.

Asking the right question…

Keep hearing discussion discussion on my social networks about whether or not global warming or global climate change is real.

Does it matter?

Isn’t the right question: Are humans negatively impacting our environment and are there things we could do differently?

Use Google Apps as Hard Drive

Over the next few weeks Google is rolling out the ability to upload all file types to the cloud through Google Docs, giving you one place where you can upload and access your key files online.

Google Docs now supports files up to 250 MB in size, which is larger than the attachment limit on most email applications, you’ll be able to backup large graphics files, RAW photos, ZIP archives and much more to the cloud.

Now instead of carrying a USB drive, you can now use Google Docs as a more convenient option for accessing your files on different computers.

Google is taking a big step to be at the center of everyone’s online office.

Update (02-07-2010):  I started using Gladinet to map a network drive to my Google and Google Apps accounts.  It allows you to quicly access on your Google Accounts.  I stopped running my backup on my office folders and just centralizing them on my personal and business Google Docs

Google I/O 2010 Open for Registration

We launched registration for Google I/O this morning.

The third annual developer conference will be at Moscone West in San Francisco on May 19-20, 2010

You can register for Google I/O here.

Checkout the schedule, sessions, and more at:  http://code.google.com/events/io/2010/

The event was a blast last year with the Android Phone giveaway and the Google Wave Announcement during the keynote.

OpenSecrets Open Data

I love data.  When I find great sources of data I have to record it.

Here is RAW and API data from the Federal Government.

OpenSecrets Open Data

It has:

• Campaign Finance – PAC and individual contributions to, and expenditures by, Candidates, PACs and Party Committees
• Lobbying – Clients, registrants (generally lobbying firms), lobbyists and bills lobbied
• 527s Data – Committees, Receipts and Expenditures
• Personal Finances – Agreements, Assets and Liabilities, Outside Compensation and Income, Honoraria and Gifts, Positions, Transactions, Travel
• Resource tables such as Category Codes, Expense Codes, Members of Congress, etc

Good stuff.   I will start working through this data.

Where Does My Money Go?

I found a much more colorful visual of where government spending goes.   It is in the UK, but still very cool.

They did a much better job visually displaying the data then I did on my Oregon or California State Budgets.

Where Does My Money Go?

Pretty cool visualizations of Government spending.  I predict much more to come in 2010.

SDK for Google Android 2.1

Google released the SDK for Google Android 2.1 today.   I need to play with the new release.

Google Fusion Tables and Data 2.0

I pulled this from the Google Research Blog.  I was going to write my own, but they said it all.  I just modified a little:

Database systems are notorious for being hard to use. It is even more difficult to integrate data from multiple sources and collaborate on large data sets with people outside your organization. Without an easy way to offer all the collaborators access to the same server, data sets get copied, emailed and ftp’d–resulting in multiple versions that get out of sync very quickly.

Google Fusion Tables is  an experimental system for data management in the cloud. It draws on the expertise of folks within Google Research who have been studying collaboration, data integration, and user requirements from a variety of domains. Fusion Tables is not a traditional database system focusing on complicated SQL queries and transaction processing. Instead, the focus is on fusing data management and collaboration: merging multiple data sources, discussion of the data, querying, visualization, and Web publishing. We plan to iteratively add new features to the systems as we get feedback from users.

You can upload tabular data sets .  Rright now, they are supporting up to 100 MB per data set, 250 MB of data per user and you can share them with your collaborators or with the world. You can choose to share all of your data with your collaborators, or keep parts of it hidden. You can even share different portions of your data with different collaborators.

When you edit the data in place, your collaborators always get the latest version. The attribution feature means your data will get credit for its contribution to any data set built with it. And yes, you can export your data back out of the cloud as CSV files.

You can filter and aggregate the data, and you can visualize it on Google Maps or with other visualizations from the Google Visualization API. You can then embed these visualizations in other properties on the Web (e.g., blogs and discussion groups) by simply pasting some HTML code.

The power of data is truly harnessed when you combine data from multiple sources.  Fusion Tables enables you to fuse multiple sets of data when they are about the same entities. In database speak, we call this a join on a primary key but the data originates from multiple independent sources.

But Fusion Tables doesn’t require you and your collaborators to stop there. What if you don’t agree on all of the values? Or need to understand the assumptions behind the data better? Fusion Tables enables you to discuss data at different granularity levels — you can discuss individual rows or columns or even individual cells. If a collaborator with edit permission changes data during the discussion, viewers will see the change as part of the discussion trail.

I don’t know about you guys but I think Fusion Tables is a game changer.  It will democratize the company database that us IT guys have made complex and hidden from others.

Many organizations actually run off series of departmental, program and other spreadsheets.   Google Fusion Tables is a great way to bring this all on the web securely and increase company access to data.   While putting the data in the hands of the people who truly own it.

URL:  http://tables.googlelabs.com/public/tour/tour1.html

Open Source Government Data Marketplace

Someone needs to create an open source version of data.gov.

Every  city, county and state municipalities need a copy.  There needs to be a well written, secure and standards based open source version available.

It would be setup identical with:

• RAW Data Catalog with XML, CSV, KML, etc data files
• Tool Catalog or Marketplace
• GeoData Catalog

It would also give all the links to integrate and share data with other government entities.

It would have an FAQ, Share Tools, Blog , etc.

It wouldn’t be that difficult to develope.  Would just need the right specifications and team behind it to make sure it is written securely.

Social Budgets and Contributions

Many projects I do in my spare time are purely for the exercise and learning experience.  I have been developing a tool I am calling social budgets.

I am very political, but I get frustrated when I don’t fully understand what is going on at a state and federal level.    I like to have a deep understanding before I take action or even open my mouth.

So I wanted to understand where the State of Oregon is spending their money and create a way I can voice my opinion on my priorities.

So I created an Oregon Social Budget site.  The next day I saw California released their budget,  so I wanted to evaluate their budget and created a California Social Budget site.

What did I learn?

• I learned about the ways that Oregon and California track their budget.
• How they spend and prioritize these budgets.
• I learned that even though governments are using the word “transparency”, they don’t actually make it any easier to get at the data.
• I learned about new agencies and commissions that operate at the state level.

It has been fun so far.  I am pushing my knowledge of government operations and building a cool little social tool.

I want to keep working on the tool and evolve it, so I am looking into other potential uses.

• Federal Budget Priorities
• Congressional and Senator Contribution Tool

Any other ideas?  We’ll see how much time I have.

Powered by Wordpress | WP Bliss theme by WP Remix
Copyright 2007. Kin Lane. All rights reserved

• Home
• About
• Cloud Computing
• Contact
• Event / Conference Technology

pageTracker._initData(); pageTracker._trackPageview(); } catch(err) {}