Google App Engine Blog

Some small updates!

2008-07-24T11:21:00.000-07:00

We're happy to announce we've released some small updates to Google App Engine. Among the more significant changes:

More apps: Want to create more than 3 applications with your App Engine account? Now you can now create up to 10!
Time windows for Dashboard graphs: Zoom in on the data in your dashboard to get a more accurate picture of whats going on. You can zoom in to see graphs for the last 24, 12, and 6 hour periods.
Logs export: You can now use appcfg.py to download your application's logs in plaintext format. Use appcfg.py --help for more information on how to download your logs.
Send email as logged in user: If you're using the users API, you can now send email from the email address of the currently-logged-in user.

Be sure to update your SDK and check the release notes for a full list of changes. Have more changes you'd like to see with App Engine? Let us know in our Google Group!

Posted by Paul McDonald, App Engine Team

Google Code Jam up and running on App Engine

2008-07-17T13:23:00.000-07:00

Google Code Jam is a coding competition in which professional and student programmers are asked to solve complex algorithmic challenges in a limited amount of time. The contest is all-inclusive: Google Code Jam lets you program in the coding language and development environment of your choice. The Top 500 participants will advance to onsite competitions at a local Google office to compete against those in their region (Asia Pacific; Europe, Middle East and Africa; and the Americas). The Top 100 will participate in the final round at the Google Headquarters in Mountain View, California.

The application that runs the competition is built on App Engine. It was developed by a few Google engineers in their 20% time and is currently serving about 200 requests per second.

If you are interested in participating you still have a few hours to register. Go to the Code Jam site and check out the live scoreboard!

Posted byPaul McDonald, App Engine Team

App Engine Community Update

2008-07-14T10:59:00.000-07:00

Posted by Marzia Niccolai, Google App Engine Team

Having spent the past two weeks tracking a hamster across America, creating collaborative pictures, and planning plenty of parties, I think it's time to get together another community update.

We were excited to find that one of our developers has started tracking App Engine performance, and hope to see more along these lines in the future. Developers continue to contribute information relating to the bulk loader, including an article that discusses UTF-8 and HandleEntity. And finally, Adobe AIR and Flex developers might be interested in this article about creating a photo application on Google App Engine.

Those of you who aren't content to just read may wish to get your hands dirty with some Open Source App Engine code. We've had 10 more additions to our Open Source project page, including projects that will interest all you Scrabble players, radio listeners, and RESTful developers.

Lastly, the forum discussion continues to keep pace, with topics that include IP geocoding, domain configuration, and continued discussion about the cap on indexed properties for an entity.

Keep letting us know what you've been up to!

App Engine Hack-a-thon: Chicago

2008-07-10T10:37:00.000-07:00

Posted by Marzia Niccolai, Google App Engine Team

Please join us on July 31st for a Google App Engine Hack-a-thon in the Windy City.

Learn about Google App Engine

The events will include training on the major features of Google App Engine, including data modeling, the App Engine APIs, some aspects of Django, and how to mashup App Engine with other web services. Google Engineers and Product Managers will be on hand to help and to answer questions throughout the day.

For a better idea of what goes on at our hack-a-thons, see our previous posts about the New York and San Francisco hack-a-thons.

Build With Us, or Build Your Own

Throughout the day, we will be building a complete App Engine application, and sharing the code with you so you can code along with us. If, on the other hand, you already have a great idea for what to build, bring that idea along. Even better, bring along anything you can prepare ahead of time (sketches, designs, web page mock ups, etc.) and use the time and information provided to develop your idea into a working application, then share it with the world.

At the end of the day, we'll invite you to share your App Engine applications with the group.

What Do I Need?

We will provide facilities, power, food and refreshments and experts to help you learn to use Google App Engine and write your application. Just bring your laptops, ideas and enthusiasm to complete the mix.

When and Where?

The Chicago hack-a-thon will take place Thursday July 31st, 2008 from 10AM-6PM. It will be held in Google's Chicago Office at 20 W. Kinzie St.

Space is limited so Sign Up now.

App Engine Community Update

2008-06-25T13:11:00.000-07:00

Posted by Marzia Niccolai, Google App Engine Team

Over the past two months we've seen the App Engine community do some exciting things. To highlight community participation, we'll be posting regular App Engine community updates.

Applications

Below are some of the more popular ones we've seen:

Wordle - Create word clouds for user provided text
Guess-a-Sketch - Fans of Pictionary will love this sketching-guessing game!
BuddyPoke - An OpenSocial application for interacting with your friends

Developer Articles

Many developers have written useful articles tutorials about App Engine. This week, we published our first externally written article on the App Engine developer pages Configuring Eclipse on Windows to Use With Google App Engine. Below are some other contributions we've found out there on the world wide web:

Google Engineers are passionate about unit tests, and it's exciting to know our developers are too! App Engine fan wrote an article 'Unit tests for Google App Engine apps'.
Thomas Brox Røst wrote an article that outlines porting the data of an existing Django App to App Engine.
A big theme for App Engine at Google I/O was creating applications that scale. The developers at Logilab attended one of the talks and wrote up a very helpful summary on Building Scalable Web Apps.

using App Engine to create a mobile website

Open Source Projects

Recently, we invited developers to list their Open Source projects on our group's pages. So far, we have 14 projects - these range from App Engine utility libraries, web frameworks, Open Source applications, and even an HTTP implementation of Map Reduce. Please continue to let us know what you are working on!

App Engine Group

Lastly, discussion continues to thrive on our Google group. Recently popular threads include discussions on full text search, session management, downloading apps, and executing common functionality before serving a request.

Google I/O session videos posted with slides

2008-06-15T18:33:00.000-07:00

Posted by Pete Koomen, Google App Engine Team

On May 28th and 29th the Google Developer Team hosted Google I/O, a developer gathering in San Francisco. For anyone who wasn't able to join us in person, you can now now catch videos of more than 70 talks, along with slides from each of the presentations, online. Among these are some great sessions from members of the Google App Engine Engineering team:

Best Practices - Building a Production Quality Application on Google App Engine
Ken Ashcraft, Software Engineer
Building Scalable Web Applications with Google App Engine
Brett Slatkin, Software Engineer
Engaging User Experiences with Google App Engine
Lindsey Simon, Web Developer, and John Skidgel, Senior Interaction Designer
Rapid Development with Python, Django, and Google App Engine
Guido van Rossum, Software Engineer
Under the Covers of the Google App Engine Datastore
Ryan Barrett, Software Engineer
Working with Google App Engine Models
Rafe Kaplan, Software Engineer

There are also a lot of interesting talks on Android, Gears, AJAX Search API, Gadgets, GData, OpenSocial, and many other Google Developer products. Feel free to discuss in our group!

Announcing Open Signups, Expected Pricing, and two new APIs

2008-05-28T11:08:00.000-07:00

Posted by Paul McDonald, App Engine Team

When we launched Google App Engine to 10,000 developers in early April, we were looking for feedback from the community. Since then, even though we've since expanded to 75,000 developers, the request we've heard most often has been to open up the preview release and give more people the chance to use it to deploy their apps to Google's scalable infrastructure. With more than 80,000 of you still waiting to use App Engine we're especially pleased today to announce that we are opening the flood gates--now anyone who wants to use App Engine can give it a try! So, if you haven't already, head over to our Admin Console and create your first application. Note: you'll need to verify your account with a mobile phone before doing so.

App Engine will always be free to get started, and we plan on enabling developers to purchase more computing resources sometime this year. Although we're not ready to offer this ability now, we've been asked by many developers to provide some insight into how we'll be pricing App Engine usage for applications that have exceeded the free quota of 500 MB of storage and around 5M pageviews per month. We'd like to be transparent about this, and have announced today that developers can expect to pay:

$0.10 - $0.12 per CPU core-hour
$0.15 - $0.18 per GB-month of storage
$0.11 - $0.13 per GB outgoing bandwidth
$0.09 - $0.11 per GB incoming bandwidth

Likewise in response to your feedback we are also providing 2 new APIs, the image manipulation API and the Memcache API.

The image manipulation API enables developers to resize, rotate, flip, crop and enhance images using the same infrastructure used in Picasa Web Albums. With the images API you can perform common image related tasks like thumbnailing images via a simple interface. For more information about the images API, please see our documentation.

The Memcache API is a high-performance caching layer designed to make page rendering faster by reducing datastore queries and improving the performance of your application. The Memcache API was written by Brad Fitzpatrick, the same developer who created memcached, a caching layer that is used in some of the world's most popular web applications, and is compatible with memcached implementations. Please see our documentation for more information about the Memcache API.

These features and improvements were motivated by your feedback and participation in our development process. We're working hard to respond to all of your requests so please continue to share them with us!

San Francisco Hack-a-thon

2008-05-19T18:07:00.000-07:00

Posted by Marzia Niccolai, App Engine Team

On Friday, May 16th, we had our second Google App Engine Hack-a-thon at the Google offices in San Francisco. For 12 hours, developers coded, talked with the Google App Engine engineering team and each other, and filled up on Google snacks.

At the end of the day we had some great results:

The beginnings of a distributed datastore helper library
http://cp.appspot.com - An online shopping cart integrated with Google Checkout
http://gawsh.appspot.com/ an application that helps generate workers compensation forms
And lastly, http://fat-burn-challenge.appspot.com/ an application to keep track of your running progress (for those of us who had too many snacks :)

Check out some pictures of the Hack-a-thon:

If you still haven't got a chance to meet the App Engine developers and build some apps with other App Engine developers, we will be holding some codelab sessions at Google I/O, our annual developer event on May 28th and 29th.

App Engine Launcher for Mac OS X

2008-05-15T11:23:00.000-07:00

Posted by John Skidgel, Senior Interaction Designer

While I was writing scenario applications to test Google App Engine, I had the following idea: If BBEdit, Dreamweaver, CSSEdit, and TextMate were at a party, what kind of application would be welcomed to help with App Engine development? As silly as this sounds, it led me to think more about workflow. I used:

a few applications for Python, CSS, HTML, and JavaScript.
the command line to run applications and to view logs.
a browser to test locally and to access the local developer console.
the command line to deploy the application.
a browser to view a live application's dashboard.

I began to think about how unfriendly command line interfaces can be, and how much repetitive typing I had been doing to test and deploy applications. With that said, I worked with John Grabowski of the Google Mac team and Brett Slatkin, an engineer on App Engine, on a 20% project to make Google App Engine Launcher for Mac OS X.

Now, App Engine Launcher is not a replacement for your code editor or your IDE. It improves your App Engine development experience by managing a list of your applications. With a few clicks you can run, browse, deploy, and view logs for your applications. It also has some added sugar thrown in like drag and drop, integration with your editor, and quick links to the local developer console and the live application dashboard.

Download it now. If it's missing a feature that you'd like to see (like scripting, a Windows or Linux version, human-level AI ;), let us know in the Discussion Group.

Google App Engine New York Hack-a-thon

2008-05-08T08:38:00.000-07:00

Posted by Marzia Niccolai, App Engine Team

Yesterday, Google App Engine held our first ever hack-a-thon at the Google Office in New York City. We had a great turn out of App Engine developers. Some programmed along with us, building a wiki, and some developers brought along their own ideas and spent the day building some awesome applications.

At the end of the day, a few developers even came up and presented their work:

An application that provides 'a simple API for writing and reading small chunks of data from anywhere' : http://tinydb.org/
An app that adds photos to the developer's facebook account
An Open Source Django utility project: http://code.google.com/p/django-ae-utils/
An application that searches for clinical trials based on keywords
An application that uses GWT, Maps and Gears to track and map appointments and uses App Engine to keep the client side data synced
An app where users can list items for sale and get emails from interested buyers: http://sellstuff.appspot.com/

Check out the pictures for the event at: http://picasaweb.google.com/appengine.hackathon/GoogleAppEngineHackathon#

We look forward to seeing you at the San Francisco Hack-a-thon on May 16th

An Open Source App: Rietveld Code Review Tool

2008-05-06T11:59:00.000-07:00

Posted by Guido van Rossum, Software Engineer

My first project as a Google engineer was an internal web app for code review. According to Wikipedia, code review is "systematic examination (often as peer review) of computer source code intended to find and fix mistakes overlooked in the initial development phase, improving both the overall quality of software and the developers' skills." Not an exciting topic, perhaps, but the internal web app, which I code-named Mondrian after one of my favorite Dutch painters, was an overnight success among Google engineers (who evidently value software quality and skills development :-). I even gave a public presentation about it: you can watch the video on YouTube.

I've always hoped that we could release Mondrian as open source, but so far it hasn't happened: due to its popularity inside Google, it became more and more tied to proprietary Google infrastructure like Bigtable, and it remained limited to Perforce, the commercial revision control system most used at Google.

Fortunately, now that I work on the Google App Engine team, I've been able to write a new web app that incorporates many ideas (and even some code!) from Mondrian, and release it as open source. The Python open source community has been trying out Rietveld for the past few days, and has already been using it to do code reviews for Python (as well as providing valuable feedback in the form of bug reports and feature requests). Of course, the tool is not language-specific: you can use it for code reviews for any language!

To learn more about Rietveld, try it out, or take a look at the code, check out the article on Google Code!

Tips on writing scalable apps

2008-04-28T22:36:00.000-07:00

Posted by Ken Ashcraft, Software Engineer Google App Engine makes writing scalable apps easier, and a well-designed app should be able to grow automatically from one user to millions. What does it mean for a Google App Engine application to be "well-designed," though? Here are a few simple tips that, if you design with them in mind, should help ensure your App will stand up with grace and aplomb, even under heavy load.

Avoid contention on datastore entities. If every request to your app reads or writes a particular entity, latency will increase as your traffic goes up because reads and writes on a given entity are sequential. One example construct you should avoid at all costs is the gobal counter, i.e. an entity that keeps track of a count and is updated or read on every request. There are some interesting ways of simulating this behavior that don't require reads/writes on every request, and we'll talk about a handy way to cache entities for reads below.
Avoid large entity groups. Any two entities that share a common ancestor belong to the same entity groups. All writes to an entity group are sequential, so large entity groups can bog down popular apps quickly if there are a lot of writes to that group. Instead, use small, localized groups in your design.
Write sparingly. Writes are more expensive than reads; keep this in mind when designing your data model. If you can avoid a write--it's best to do so.
Define a main() function for code reuse. Instances of your app are kept running for a certain period after each request, so there's a chance that any request will hit an already-running app. Apps that define main as in the following example:

def main(): application = webapp.WSGIApplication(_URLS, debug=True) wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main() can reuse global variables, so although there's no guarantee that any given request will hit a running app instance, you can take advantage of this to non-deterministically cache entities and other expensive values. This works simply because App Engine will detect if main() is present in an already-loaded script and call it instead of re-importing.
Profile your code. You can use the Python profiler to profile your code's performance.

Do you have tips for scaling web apps on App Engine? Post them in our Google Group!

Google App Engine + Google Data APIs: A Match Made on the Web

2008-04-23T16:59:00.000-07:00

Posted by Marzia Niccolai, Google App Engine Team

Don't you think it'd be cool if you could blog about a party, add it to your calendar and invite all your friends in one fell swoop? After the party, how about viewing photos and videos from the event in the same application? The Google Data APIs make all this possible, and your Google App Engine apps can use feeds from Google services to display and update information for a particular user stored by these services.

The Google Data APIs team just released an update to the Google Data Python client library which includes support for Google App Engine. With this library, your app can now use AuthSub to gain access to your users' feeds.

Jeff Scudder, member of the Google Data APIs team and the main contributor to the Google Data Python client library, has written an article that takes you through all of the steps necessary to use Google Data feeds in your Google App Engine application. He's even added his sample application to our Google App Engine sample code project.

Try out it, and let us know what you come up with!

Google App Engine Hack-a-thons!

2008-04-21T18:26:00.000-07:00

Posted by Dick Wall, Google App Engine Team

In May, Google will be holding two free Google App Engine developer events: one in New York and one in San Francisco.

Learn about Google App Engine
The events will include training on the major features of Google App Engine, including data modeling, the App Engine APIs, some aspects of Django, and how to mashup App Engine with other web services. Google Engineers and Product Managers will be on hand to help and answer questions throughout the day.

As well as covering the Google App Engine SDK, we will show you how to deploy and manage your own applications on Google App Engine servers. For people who don't already have a Google App Engine account, we will be inviting people who come to the hack-a-thon to develop on App Engine.

Build With Us, or Build Your Own
We will use the information shared at the event to build up a working Google App Engine application throughout the day, which you can code along with us. On the other hand, if you already have a great idea for what to build, bring that idea along. Even better, bring along anything you can prepare ahead of time (sketches, designs, web page mock ups, etc.) and use the time and information provided to develop your idea into a working application, then share it with the world.

What Do I Need?
We will provide facilities, power, food and refreshments and experts to help you learn to use Google App Engine and write your application. Just bring your laptops, ideas and enthusiasm to complete the mix. We will assume some basic skills and preparation for the event, and these details will be provided soon.

When and Where?
The first event will be in New York, on May 7th from 10am to 6pm. The second will be in San Francisco on May 16th from 10am to 10pm. Space is limited so sign up now!

New York Hack-a-thon
San Francisco Hack-a-thon

Google App Engine Helper for Django

2008-04-18T13:14:00.000-07:00

Posted by Matt Brown, Google App Engine Team

Since the launch of Google App Engine one of the most common requests has been for better integration with the popular Django framework.

We've been working hard and we've made available a helper that eases the process of developing a Django project to run on Google App Engine.

The helper will initially provide you with:

The ability to use most manage.py commands
A BaseModel class that appears the same as the standard Django Model class.
The ability to serialize and deserialize model instances to JSON, YAML and XML.
Access to Django's test framework with a test datastore and support for fixtures.

We are releasing the helper under the Apache 2.0 license. We look forward to working with the Django community to add further functionality as time goes on.

You can download the helper from http://code.google.com/p/google-app-engine-django. Issues and Feature Requests should be added to the tracker within that project.

Getting your data on, and off, of Google App Engine

2008-04-14T22:53:00.000-07:00

Posted by Kevin Gibbs, Software Engineer

One upcoming feature of Google App Engine that's worth mentioning is large-scale data import and export. We know that many of you have large apps that already run on other platforms, and getting all of that data into the Datastore API is a challenge. And, conversely, we also know that a lot of you want to make sure it's easy to move off Google App Engine in the future, in case you need to switch to a different platform.

With Google App Engine, you own all the data in your app. As stated in our terms, you always have the right to get your data out of Google App Engine at any point. We wouldn't have it any other way.

To that end, we would like to solicit feedback on what form you would like the data exporter feature to take. We've started a thread in the discussion group about this, and we'd love to hear what you think. XML output? CSV transform? AtomPub? RDF? Let us know!

We're up and running!

2008-04-11T10:49:00.000-07:00

Posted by Pete Koomen, Product Manager

Google App Engine has been live for several days now and we're thrilled with the response we've gotten so far! We filled our initial batch of accounts within a few hours after Monday night's Campfire One. And today, we've just invited another 10,000 developers to try it out.

We're excited to see the beginnings of a developer community forming in our Google Groups! We're actively reading your posts and responding with help where we can, and it's great to see some of the more experienced members of the community start helping others out too!

We've also seen a lot of feature and bug reports on our Issue Tracker, including requests for Ruby, Java, Perl, PHP, and Fortran. Here are are some of the general areas we're focusing on right now:

Support for more languages. We're obviously huge Python fans, but we recognize that there are other great languages out there that developers use to build web applications.
Support for offline processing. Right now Google App Engine is great for web apps that do all of their processing in response to user requests, but what about apps that need to perform scheduled tasks or larger-scale data migration? We'd like to support those apps too.
Support for large files. Google App Engine currently imposes a limit of 1MB on all requests, both inbound and outbound. We're interested in providing efficient support for much larger uploads and downloads.
Billing for additional quota. During the preview release period, all apps are restricted to a set of free resource quotas. Although Google App Engine will always be free to get started, we plan on allowing developers to purchase additional resources in the future, while paying only for what they use.

Judging by the the feedback we've gotten so far, these are areas you're interested in as well--we'll keep you posted. Keep letting us know what you think about the Google App Engine and where you think it should be headed!

Introducing Google App Engine + our new blog

2008-04-07T21:17:00.000-07:00

Posted by Paul McDonald, Product Manager

At tonight's Campfire One we launched a preview release of Google App Engine -- a developer tool that enables you to run your web applications on Google's infrastructure. The goal is to make it easy to get started with a new web app, and then make it easy to scale when that app reaches the point where it's receiving significant traffic and has millions of users.

Google App Engine gives you access to the same building blocks that Google uses for its own applications, making it easier to build an application that runs reliably, even under heavy load and with large amounts of data. The development environment includes the following features:

Dynamic webserving, with full support of common web technologies
Persistent storage (powered by Bigtable and GFS with queries, sorting, and transactions)
Automatic scaling and load balancing
Google APIs for authenticating users and sending email
Fully featured local development environment

Google App Engine packages these building blocks and takes care of the infrastructure stack, leaving you more time to focus on writing code and improving your application.

Today's launch is a preview release -- we're by no means feature-complete, and we're giving you early access because we really want your feedback. This preview of Google App Engine is available for the first 10,000 developers who sign up, and we plan to increase that number in near future.

During this preview period, applications are limited to 500MB of storage, 200M megacycles of CPU per day, and 10GB bandwidth per day. We expect most applications will be able to serve around 5 million pageviews per month. In the future, these limited quotas will remain free, and developers will be able to purchase additional resources as needed.

If you'd like to try it out, sign up for access to the preview release and then download the SDK to get started. The best place to give us feedback is in the discussion group. Future announcements from us will appear there and/or this blog.