I have been reviewing a lot of APIs lately. I”m evaluating their potential for integration into applications.
As I’m going through I”m also evaluating how they deliver their API community.
I was checking out the Scribd API tonight, and I like their delivery. Its pretty straightforward.
It has the following API building blocks:
- API Overview
- Sign Up for an API Account
- API Tutorials
- Client Libraries
- User Authentication Overview
- Usage Limits
- Javascript API
- Mobile and E-Reader API and SDK
- Enterprise SDK
- Help / Support
- FAQ
They make it easy to sign up for an API account by providing links in navigation and on home page.
They provide client libraries in these programming language:
- Ruby
- Java
- PHP
- C# / .NET
- ActionScript 3.0
- Python
They just make it easy to get up and going by providing the necessary information and tools.
Its a good example to follow when deploying an API.
Tags: .NET Framework, Application programming interface, Java, JavaScript, Languages, PHP, Programming, Scribd
I finally got my Google Storage for Developers email invite the other day. They promised attendees of Google I/O would get them quicker, it took 3 months.
I clicked on the link in my email and requested my keys and logged into the Google Storage Manager.
It has a pretty basic interface for managing your buckets and objects. Just like Amazon S3.
Then I wanted to test out the interoperability of it. Google claims:
Google Storage is interoperable with a large number of cloud storage tools and libraries that work with services such as Amazon Simple Storage Service.
So I downloaded a common Amazon S3 PHP Class and changed the request endpoints, request headers, signature identifiers, ACLs, query string parameters and used my Google developer keys.
And it work. It listed my buckets I had created through the Google Storage Manager. I am playing with more to see how I can store files and content at both Amazon S3 and Google Developer Storage.
Now to figure out what I will use Google Storage for vs. Amazon S3.
Tags: Amazon S3, Cloud Storage, Google, Google I/O, Google Storage
I’m spending time going through RedHat’s Deltacloud set of cloud APIs. They recently recently moved the project to the Apache Incubator and submitted it to the Distributed Management Task Force (DMTF) .
With the Deltacloud API you can start an instance on an internal cloud, then with the same code start another on Amazon EC2 or Rackspace. This seems like the closest thing we have at the moment, to a true cloud API standard.
Deltacloud Core provides:
- REST API
- Backward compatibility across versions
Deltacloud Aggregator provides a web UI in front of the Deltacloud API. With Deltacloud Aggregator, you can:
- View image status and stats across clouds, all in one place
- Migrate instances from one cloud to another
- Manage images locally and provision them on any cloud
Deltacloud API and Aggregator are free and open source and support the following providers:
They will also soon be supporting:
They offer up a Ruby Gem for download. I will have to get a new environment up and running so I can play with. The ability to manage resources across multiple cloud providers and allowing for instance portability is very important to me. Hopefully I can get time to play with more.
Tags: Amazon EC2, Apache Incubator, Cloud APIs, Deltacloud, Distributed Management Task Force, GoGrid, Open source, Rackspace, Red Hat
In addition to adopting Gist, I have quickly adopted a new tool that shows you rich contact profiles right inside Gmail. It is called Rapportive.
I have heard about them, and Audrey uses them, but until I watched them present at YC DemoDay it didn’t really stick.
As soon as I got home I installed Rapportive in Firefox and Chrome. Amazing!
You get so much intelligence on each contact in your Gmail inbox. I started searching for old contacts just to see what it would show.
Rapportive is an essential tool in my social CRM toolbox.
Tags: Firefox, Gist, Gmail, Google, Google Chrome, Rapportive
I have successfully integrated social CRM service Gist into my platform. I have bee using it on a daily basis for about a month now.
Gist allows me to integrate my:
- Personal Gmail account and Google Apps Gmail
- Personal Google Contacts and Google Apps Contacts
- Google Calendar
Gist then gives me a robust, yet intuitive dashboard to manage people and companies I interact with on a daily basis.
I can set the importance of each contact and view social streams, blog posts, emails, and events for all my contacts.
Gist gives me all these great tools for managing my CRM as well as bringing in tons of social and other contact related data.
Only thing I don’t like is that I really don’t have a public facing job at the moment, so my contacts list is very stagnant. That is not really Gists fault though.
Tags: Cloud Computing, Facebook, Gist, Google, Google Calendar, LinkedIn, Twitter
CloudCommons has developed the Service Measurement Index (SMI) Framework, a set of tools for comparing cloud computing service providers.
The Service Measurement Index (SMI) Framework uses the following metrics:
- Quality - There are often many different solutions to any IT challenge, and each solution has different quality-of service-attributes, such as defect rates, time to repair, resiliency and scalability. IT needs to be understood in terms of how well it provides quality, in business-relevant and granular terms.
- Agility - In today’s world, new technologies and approaches emerge at a dizzying pace, and IT must become nimble and agile if it is to take advantage of these developments. Agility speaks both to how quickly new capabilities are integrated into the service as needed by the business, and how easily the business can adopt different services or technologies.
- Risk - No organization can tolerate excessive risk, and yet the risk picture at any point in time is one of the least understood components of IT. Risk is multi-dimensional, ranging from obvious sorts of risks – the business stability of a key provider – to the less obvious risks, such as the effect of business ethics lapses on a provider’s customers.
- Cost - As a direct measure of efficiency, cost is of vital importance to IT and the business. Cost tends to be the single most scrutinized metric today, but it is important to express cost in terms relevant to the business. Cost must therefore be understood and allocated to IT services in a granular, business-centric way that provides transparency into the overall IT operation and relates to value.
- Security – Data protection and privacy is important to nearly every organization, but some have more stringent security requirements than others. Financial organizations, for example, have compliance regulations involving data integrity and privacy that they must meet. Security is multi-dimensional, and includes data retention/disposition, accountability, privacy, data loss, and integrity, among other elements.
- Capabilities - The definition of a given service can be found in the list of essential capabilities that it provides; an email service must be able to send and receive email and an e-commerce service must be able to manage orders. These key capabilities can be evaluated for completeness or user satisfaction, and joined by a larger set of essential other capabilities to further differentiate between various providers.
I’m very optimistic about where the cloud computing industry is going with standards like this for comparing cloud service providers and the DeltaCloud Cloud API becoming a standard.
Tags: Business, Cloud Computing, Security, Standards
I am needing more programmatic control over my Amazon EC2 environment. I am rebuilding all server instances on my Amazon Web Services network.
I needed to reconfigure using EBS Volumes for file and data storage, and upgrade the operating systems and some applications while I’m at it.
So I’m creating all new AMI as well as reconfiguring my snapshot strategy. The central object I use for programmatic manipulation of Amazon was limited.
I need to add some methods for managing the Amazon EC2 Volumes:
- CreateVolume
- DescribeVolumes
- AttachVolume
- DetachVolume
- DeleteVolume
I’m going to need to automate the create of snapshots as well, so I added the following methods:
- CreateSnapshot
- DescribeSnapshots
- DescribeSnapshotAttribute
- DeleteSnapshot
This will give me additional control beyond just creating and managing instances. With my new EBS framework I have more granular control over different file and data stores for taking snapshots, etc.
With this more granular control over drives and their files and data, plus I was just able to offload a lot of “heavy files” like audio, flash, pdf, images, etc. storing all the files at Amazon S3.
Tags: Amazon Elastic Compute Cloud, Amazon Web Services, Data, Servers
I once quit a job because I got called into a meeting where the agenda was to discuss having too many meetings.
I had a chance to play with the AlchemyAPI I came across today. AlchemyAPI is a semantic tagging and text mining Application Programming Interface (API).
I have about 10K web pages I want to extract top keywords and key phrases from. I want meaning extracted from the words on each page.
AlchemyAPI provides nine methods:
- Named Entity Extraction – Identifies people, companies, organizations, cities, geographic features and other entities within content provided.
- Topic Categorization – Applies a categorization for the content provided.
- Language Detection – Provides language detection for the content provided.
- Concept Tagging – Tagging of the content provided.
- Keyword Extraction – Provides topic / keyword / tag extraction for the content provided.
- Text Extraction / Web Page Cleaning – Provides mechanism to extract the page text from web pages.
- Structured Content Scraping – Ability to mine structured data from web pages.
- Microformats Parsing / Extraction – Extraction of hCard, adr, geo, and rel formatted content from any web page.
- RSS / ATOM Feed Detection – Provides RSS / ATOM feed detection in any web pages.
I’m only using the keyword extraction and named entity extraction for what I am doing. The whole API provides some great tools to quickly harvest, scrape and process content from the open Internet.
Their API is extremely easy to use and you can be up and running in about 10 minutes harvesting and processing pages.
Tags: Application programming interface, Harvesting, Keywords, Language, Online Tools, Parsing, Scraping, Screen scraping, Text mining
I have a “hobby” harvesting project going on right now. I’m am harvesting several thousand web pages from various locations on a certain topic.
I’m not really interested in the content itself, but the top keywords that are used in the content. I pulled the page and extracted the body content that I am interested in analyzing.
I then want to pull a list of the top keywords and key phrases used in the page.
So I am faced with next steps:
- Spend time researching algorithms for parsing text and looking for words
- Write my own scripts for identifying key words and key phrases (harder than you think)
- Evolve and continue scripts to make more intelligent over time
There is a lot of time and money investment there. Easy to give up on my project at this point.
I start looking for existing tools or APIs that would accomplish this. I came across a great tag, keyword, and key phrase intelligence API called AlchemyAPI.
They have a great API that takes content, and returns an intelligent set of tags. Now I am one more step closer to being able to extract meaningful key phrases and tag sets.
It is also a great example of how good Application Programming Interface (API) can assist developers in delivering projects.
Tags: Application programming interface, Harvesting, Intelligence, Keywords, Parsing, Scraping, Tag Clouds, Tags
I can parse any document or web page and tell you what words exist in the content. I can tell you which words are most used or potentially meaningful in the content.
What I would like to have is more meaningful sets of key phrases and tag sets that are specific to certain topics and industries.
I would like to be able to sent a title of a page or blog entry to an API, then have it return several tag and key phrase sets that are relevant to run against the body of the page to see if they exist.
Then I could run a secondary comparison against various keywords I find in the body.
Make sense? Yeah geeky I know. Just sharing a thought, out loud!!
Seems like an API you could build upon Google Analytics and use the TargetingIdeaService. I will have to investigate other existing keyword tools out there that have an API.
Tags: Google Analytics, Keywords, Tag, Tag Cloud, Tag Sets
I’m seeing a lot of chatter on the Internets lately about API development and best practices. Like 10 Common Mistakes Made by API Providers at RWW and APIs: an Important Part of Product Strategy at ProgrammableWeb.
I had the pleasure of sitting with the engineer from Mashery a couple weeks ago and listen to their assessment of the API market. Ever since then I’ve been reviewing their approach to API deployment and working to understand the playing field better.
Today I came across Sonoa and the free API tools at Apigee. Apigee offers:
- API Testing
- API Debugging
- API Analytics
- API Protection
I’m checking out Sonoa and what they offer more. I am also adding testing, debugging, and protection to my list of API building blocks.
Tags: Facebook, Google, Mashery, Programming, Sonoa Systems, Web APIs
A positive feedback loop can occur whenever an API becomes popular, it keeps becoming more and more popular due to network effects.
The most important network effects are:
- API gets support by partners
- API gets support by vendors
- Developers are more likely to already know how to use it.
This feedback loop and network effect are the momentum you need to create in your API community.
I’m working through this thought extracted from Why you should use OpenGL and not DirectX.
I am still processing a great post at High Scalability called, Scaling an AWS infrastructure – Tools and Patterns. They cover several tools you can use to take advantage of Amazon’s Web Service and suggest an architectural model you should adopt for a scalable infrastructure in the cloud.
They suggest the following tools for managing your amazon scaling:
- Puppet for managing your Amazon EC2 instances
- Capistrano for cloud task automation
- Centreon/Nagios, Zabbix, Cacti and Munin for cloud monitoring
- Syslog-NG for centralized log file management
They even cover several tools for optimized data access:
- Structured relational data with MySQL or a PgSQL on Amazon EC2
- Storage for more volatile data with tools like Redis, Tokyo Tyrant/Tokyo Cabinet, MemcacheDB.
There are a lot of great points on how to scale your Amazon Cloud infrastructure. Exciting to see different approaches to scaling with Amazon Ec2 and S3 and throwing in some innovative open source tools that make the process much easier and powerful.
I was reading a good post on getting your API right. They propose following these rules when developing your API:
- Use HTTP
- Use Your Verbs
- Keep Your URL/URIs Consistent
- Use Your Status Codes
- Expose (And Accept) Multiple Data Formats
- Protect Your Users with OAuth
- Don’t Shut Off HTTP Authentication Entirely
- Document, Document, Document
There is a lot more to it, but this is an excellent framework to follow when developing your Application Programming Interface (API).
Tags: Application programming interface, Hypertext Transfer Protocol, OAuth, Web APIs
I was just refining a wiki page of various building blocks I use at Amazon Web Services. I noticed it would make a great Internet Service Provider (ISP) package for someone who wanted to start an ISP, or even used as model for an existing ISP looking to migrate to cloud computing.
These are a few of the components I have my list:
- Web Server on Amazon EC2
- Linux / Windows
- EBS Volumes for Storage
- Machine Images
- Amazon S3 Central File Storage + Jungle Disk
- Server Backup
- Central File Storage
- Client Cloud Storage as a Service
- FTP Access
- Database
- SQL Server 2008
- MySQL
- Amazon RDS
- Other (Amazon SimpleDB, Cassandra, CouchDB, etc)
- Email
- POP Server
- SMTP Server
- DNS Server
- FTP Server
- Web File Access
- Central File Storage Access
- SVN Server
- Version Repositories
- Client Checkout
These are just a few of the building blocks I have on my list. There are many other possibilities and configurations. You slap on an ISP Cloud Management tool like Plesk from Parallels and you can manage your network without much headache.
You are going to see many Internet Service Providers (ISP) make the jump to the cloud because of cost and ease of deployment. There are just too many benefits to ignore the cloud.
Tags: Amazon Elastic Compute Cloud, Amazon S3, Amazon Web Services, Cloud Computing, Internet service provider, Servers, SimpleDB
I have been reviewing as many Cloud Service providers as I can get my hands on. I am building a database right now of them and their products, locations, and types of providers. I am breaking them into the following types:
- Software as a Service (SaaS)
- Platform as a Service (PaaS)
- Infrastructure as a Service (IaaS)
- Open Source
- Enterprise
- Education
- Government
Cloud service providers usually fit neatly into one or many of those categorizations without a problem. Except for one group; the growing number of cloud management software providers.
There are some that have their own data centers or platforms so they easily fit into a category. However there are many service providers that purely provide a cloud management set of tools that work with one provider (Amazon) or several providers, and they offer no infrastructure or platform. They are a software, but not the end user Software as a Service (SaaS) we are used to.
I have gone back and forth on this one, feeling they need their own categorization. Then no….they are just a IaaS provider, or wait no Paas, wait no…SaaS.
I even discussed this with Audrey and she was intrigued enough to ask RWW readers if the 3 Common Acronyms for Cloud Services Sufficient?
I am now dubbing cloud management software as Management as a Service (MaaS).
Last night I went through 25 of the top Application Programming Interfaces (API), and documented what type of services they offer as part of their API Ecoystem. I’m trying to get a handle on best approaches to delivering an API. I’m still processing this information but so far I have these API Ecosystem building blocks:
API Overview
Account / Developer Related
API / Code / Applications
Support Related
|
Resources / Informational
Other Tools / Not Categorized
|
This list represents most of the tools provided to partners, developers and API consumers through existing top APIs. It is interesting to see how different players approach this area of their business. Some its their only business.
I can’t rave enough about how Salesforce delivers their API. They are definitely the industry standard for delivering a highly sophisticated API Ecosystem with their Software as a Service (SaaS).
I am fascinated by APIs. I got to listen to the engineer from Mashery talk about their API services the other day. Ever since then I’ve been thinking a lot more about APIs. I just reviewed 25 top APIs out there:
- Flickr API
- Evernote
- Delicious
- Youtube
- Ebay
- Last.fm
- Digg
- Box.net
- Bing
- Salesforce
- Google Earth API
- Skype
- LinkedIn API
- Yahoo Developer Network
- Zillow
- Vimeo
- 37 Signals
- Tumblr
- GetGlue
- Amazon Web Services
- Eventful
- Twilio
- Shopping.com
I’ve been looking at each providers approach to delivering an API ecosystem. Lots of different approaches and common building blocks.
Google wins for the most simplest approach to deploying APIs. Salesforce definitely is the leader in delivering a very robust and complete API ecosystem, its no wonder they are the leading Software as a Service (SaaS) provider.