Building Bing Smart Search for Windows 8.1

This blog post is part of a series of guest posts we’re publishing this week from different people in groups across Microsoft who helped us build Windows 8.1. – Brandon

My name is Kieran Snyder, and I’m a Group Program Manager in the Bing User Experience team. My team was responsible for working hand-in-hand with Windows engineering to build the new Bing Smart Search in Windows 8.1. I’m excited to tell you a little bit about our work with the Windows team to build what is one of the coolest and most visible improvements to Windows 8, and highlight a few of my favorite things, now that it will be broadly available to everyone tomorrow.

Integration from the Start

The Bing/Windows collaboration on Smart Search for the Windows 8.1 update started with our work on Windows 8 when our teams worked together to build the Bing search app. The focus for that app was a cross-team effort to create a modern, fast and fluid search experience that was great for touch, and took advantage of what the revolutionary Windows 8 platform offered. Both teams brought different perspectives to the table in those first days of collaboration. Bing brought a deep knowledge of core search, including experience with experimentation and agile development, expertise in machine learning and relevance, and specialized knowledge in multimedia. Windows was embracing modern design, and brought the kind of unrelenting attention to detail and craftsmanship that is essential when you’re shipping software to a billion people around the world. And as we worked through that initial project across teams – and I was actually a Group Program Manager in the Windows team at the time – we realized how much more opportunity there was to rethink what search should be and to build something really great for people that brought all the pieces together.

From the start of Windows 8.1 development both the Windows and Bing engineering teams knew that we wanted to go all in on search – together. For me personally, the Windows 8 collaboration brought me back to my old roots in natural language processing and got me hooked on the search space, so when work began on Windows 8.1 I moved to the Bing team to learn everything I could about it. But, to make a cross-company collaborative project work at this scale, you need a truly diverse set of individuals involved. On the Bing side we started building a new team to work on Smart Search that drew from a wide variety of expertise and backgrounds. Our team now includes people with advanced graduate degrees and some straight out of college; folks with varied perspective as designers, developers, researchers and marketers; people with experience in Windows, Office, Internet Explorer, Visual Studio, as well as companies outside of Microsoft and the startup scene. And that’s just our small corner of Bing. All told, nearly 1,000 people in Bing have contributed to Smart Search in some way, including program managers, developers, testers, designers, researchers and our colleagues in business development and marketing.

Rethinking what search should and could be

The promise of creating something together – making a modern search experience that includes the best of Windows search and the best of web search all in one place was an exciting challenge. We knew we wanted to help people find what they needed, no matter where it lived or what it was. We asked ourselves, “Why should someone have to remember where they saved something? What if there is a forgotten note or picture tucked away that can be helpful; how would you know to go look for it? Shouldn’t Windows help you find all of that?” That was the underlying goal with Smart Search – let you simply swipe or type to find everything, all in one place. Whether it is a document on your PC, an app you downloaded months ago, a picture stored in SkyDrive or a website, Bing Smart Search will bring it to the surface. As you’ll see, Smart Search helps you find stuff fast wherever it lives.

A challenge with designing search that works everywhere was reconciling how people have historically searched on Windows versus how they have historically searched on the Web. Consider what we call “query formulation”, which is when you enter your search term in the search box. On the web, when you enter a query in the Bing search box, hitting enter launches a search query for exactly the string you’ve typed – like this:

But in the world of Windows when typing “sky” and pressing enter, you expect to launch SkyDrive directly, rather than searching for “sky” and getting a web search results page. It took us many rounds of iteration to make sure the Smart Search experience exactly preserved web search expectations, but didn’t compromise the efficiency of quick commanding in Windows. Notice how typing “sky” in Smart Search surfaces SkyDrive, as well as related Control Panel settings, and more:

That is just one example of the challenges we had ahead of us to rethink the search experience for a new and improved Windows experience. We quickly realized that even with a great team and a strong partnership, this wasn’t going to be easy!

Leveraging Satori for the Windows canvas

Within the varied backgrounds of our two teams were also many strengths that have given us the opportunity to look beyond the constraints of a traditional search box and create transformative experiences that are unlike anything you can find in other products. My favorite example of this is a feature we call “search heroes.”

To provide background on the search heroes I want to outline a little context on some pretty cool technology that most people don’t know about. For many years, Bing has invested in Satori, which is essentially a repository of the many people, places, events, and objects in the world – what we call entities – and more importantly the relationships between them. At last count, and this changes due to the constant changing nature of the web, Satori includes hundreds of millions of entities and understand billions of points of information linking them together. Satori provides the foundation for our Snapshot experiences on Bing.com, so that when you search for your favorite celebrity or famous landmark, you get an arrangement of facts, biographical information, images and other key information in addition to the most relevant web results.

Windows 8.1 provides a fantastic, full-bleed canvas where apps can shine, and we realized pretty quickly how the investment made in Satori could be leveraged across this canvas for a Smart Search experience that is beautiful as well as functional. Search heroes take this vast set of structured data that Bing has about the world and turns it into something that feels beautiful, curated, and relevant – something that looks and feels like the best modern apps. Thanks to Satori and Bing, you can think of these experiences as mini-apps that Windows creates on the fly. As you would expect of an experience built for Windows, they include deep links into your Windows applications to help you get stuff done.

Take music, as an example, which has been one of the most popular categories in the Release Preview. With Bing, Windows, and the Xbox team working together, we can present a big, beautiful answer that puts the most relevant information and actions front and center. We are able to include immediate music playback of all the most popular songs in the Xbox Music catalog, serve up music videos, launch apps that can help you go deeper and get more done, and surface other information that is easily accessible with a swipe of your finger. Across Windows, Xbox, and Bing, we built something that none of these teams could have made in isolation.

Dynamic and agile

There isn’t a single part of the Smart Search experience that hasn’t been built in collaboration across our teams, including our state of the art image search and our results page that combines relevant results from Windows, apps, the web, and your personal content from SkyDrive. But perhaps the most critical investment that we’ve made together is our update pipeline and flighting service, which allows Bing to deploy new search features and refinements to Windows every single day. Now you can get critical search updates every time you use Windows – versus waiting for periodic updates. With Smart Search we dynamically reflect the ever-changing web, we can learn what’s popular and what’s not, we can try out new experiences to see how well people like them, and we can improve your experience daily. This is the largest-scale investment in server-side development for Windows that Microsoft has ever made, and it took incredible commitment on the part of the Windows team to build this. More than any other part of our collaboration, this is what sets us up to make new, dynamic experiences together for years to come, delivering what people need when they need it.

One Microsoft

Building something together in such a close way is often hard at a big company. From the very beginning of this project, Katie Frigon, my counterpart in Windows, and I have had full commitment to a shared design process. For fifteen months now, every User Interface PM and designer on the project has gotten together for two hours every Friday morning to share work in progress and to critique and discuss together. These design forums are meetings we continue to have as we look to our next set of features and investments. There is just no substitute for having everyone on a project in the same room together week in and week out to make the project work. We hold shared architecture reviews and code reviews for developers. The lead designers on the project recently gave a presentation to user experience leads around the company about this design process so they can share their learnings. Windows and Bing user researchers test each other’s features in the lab and run shared studies. At all levels of the company and across all disciplines, collaboration has been open, effective, and frequent, and I’m thrilled about what this has allowed us to build together and excited about what comes next.

Smart Search is just the beginning of what’s new in Windows 8.1. We can’t wait for you to check it out! If you have Windows 8, (tomorrow) just go to the Windows Store on the Start screen and get the update. If you don’t yet have Windows 8 or are waiting for all the great devices coming this holiday season, you can head to www.bing.com/smartsearch to see it in action and get a feel for what you’re missing.

Kieran Snyder
Group Program Manager
Bing User Experience Team

View the original article here

Building on our vision of modern computing

Hello Julie and congrats for the great job you guys are doing on both Windows and Windows Phone.

I am using both Windows 8/RT across my PCs/Tablet and Windows Phone 8.

I love and strongly believe and supporting to Microsoft's new vision that introduced with the new Windows. I am following the Windows Blue development very closely and I really like what I see so far (well from the leaks) Windows 8 evolving and changes based on customer feedback and while I see you guys responding to almost all major criticism on Windows 8, I am surprised and kind of disappointed that one major subject has not been discussed, or leaked yet. It is my personal No1  complain from the Windows 8 and Windows Phone 8 and my No1 request for Blue in both Windows and Windows Phone, its causing major problems to me and thousands if not million others and its nothing else than the absent of a real Notification Center.

Currently the Notifications on Tiles method you are using is causing the following issues:

1. The only way to get Notifications is to pin a Tile to the Start Screen, so we have to pin tiles for Apps we do not even want to have in our Start Screens just to get Notifications.

2. Even then, the Notifications are spread across the Start Screen (which is getting bigger and bigger in daily bases) and we have to manually scan every single tile to spot a numeric notification on Tile.

3. Notifications are hardly synchronized across my Windows devices. I read a Skype message on my phone and its still shows on my Tablet for example until I read it again from there. That happen with many apps.

4. Lock Screen Notifications are limited to a specific number. I never had more than 7 notifications pending in my lock screen at the same time, but I always have more than 7 Apps issuing Notifications in different times and due to the App limit we cannot have proper Lock Screen Notifications, I am always missing something because of the App limit.

5. App specific Controls like Music controls are appearing randomly on the screen

So basically we have a great App oriented OS with most Apps issuing and generating Notifications and we do not have a Notification Center to handle them properly.

Like all major competing OS already offer, we also need an advance Notification Center to host and organize ALL App Notifications under one roof and be able to read, or dismiss Notifications from there easily. From the same place we can also access App Specific Controls like Music Volume etc. All App can then push the same notifications (just numeric and not in detail) on the Lock Screen, there is no need for Notifications Limit on the Lock Screen please take off this limitation.

I really really really hope you guys are working on it for BOTH Windows and Windows Phone Blue (it just didn't leaked yet)

I really hope that finally a real Notification Center will be a major part of the next update.

Keep up all the good work, I hope such a feedback will help the team even farther.

Thanks

View the original article here

Building Office for Windows RT

AppId is over the quota
AppId is over the quota

Editor’s note: Earlier this year, Steven Sinofsky announced Windows support for ARM processors (Windows on ARM or WOA). His post on the Building Windows 8 blog included the following news about Office:

“WOA includes desktop versions of the new Microsoft Word, Excel, PowerPoint, and OneNote. These new Office applications, codenamed “Office 15”, have been significantly architected for both touch and minimized power/resource consumption, while also being fully-featured for consumers and providing complete document compatibility.”

In the months since that announcement, there has been a great deal of speculation about what we are delivering. David Brodsky, Partner Test Manager and Josh Pollock, Principal Development Manager join Office Next to walk you through what Office for Windows RT is and why it exists; specific optimizations we’ve made (and why most of those will benefit you on other platforms too); what wasn’t possible and the functionality differences that resulted; and finally how to get Office Home & Student 2013 RT.

Shortly before the January 2011 Consumer Electronics Show (CES), we were asked if we could create a version of Office that would run on reference hardware with an ARM processor. This led to Steve Ballmer’s CES demo of Word and PowerPoint running on ARM. Office for Windows RT was born.

In doing research for Office for Windows RT, we spoke to people to understand how they use current, in-market tablets. We wanted to understand what was missing that would make for a more compelling experience. One answer was nearly unanimous – people wanted a complete Office experience; not just a viewer. However, they also wanted a version of Office that was optimized for the tablet form factor – most importantly supporting touch and providing long battery life.

Office Home & Student 2013 RT is Office running on the ARM-processor based Windows RT OS. It is full Office built from the same code base as the other versions of Office, with small changes that were required as a result of differences between Windows 8 and Windows RT. Our goal when starting the Office for Windows RT project was to deliver:

ARM as a “first class” platform, including the Same look and feel as x86/x64, same level of polish and reliability, full Office feature-set and fidelity and Service parity (e.g., save to SkyDrive, roaming settings, other Windows Live integration, Office.com experience, etc.)Stunning battery life (both active and idle), great performance and responsiveness

While we knew we wanted to deliver the same great Office experience across Windows and Windows RT, we also knew that Windows RT gave us an opportunity to optimize the product for the type of devices that would run on this new platform.

The Office Home & Student 2013 RT applications – Word, Excel, OneNote, and PowerPoint run on the desktop on Windows RT. Our highest priority was making sure that customers can successfully do the things that are most important to them when using Office on these Windows RT devices.

When you use the Office applications on Windows RT, you’ll find yourself in a familiar environment that feels just like what you are used to using on other platforms. You may also notice some of the enhancements we made to optimize Office for the Windows RT environment - like automatically enabling touch mode (but if you don’t notice and find that everything just works as you’d expect, that’s okay too).

When you look at the specifications for Windows RT devices, you’ll find that most share some common characteristics:

Touch-enabled; Portable with the ability to run on battery for long periods of time; Memory capabilities start at about 2GB; SSD drives of 16 to 32 GB capacity; and Wireless connectivity with some having 3G

We’ve optimized Office for these capabilities. We’ve already described our touch investments in the post “Using the new Office with touch.” In addition to these broad investments in touch, Office applications are enabled for “touch mode” by default, so the touch experience is better out of the box.

Let’s take a look at some of the other improvements we’ve made while developing Office for Windows RT.

Battery life is a difficult problem that requires good citizenship from all parts of the system to get the best possible results. A single poorly-performing component, whether hardware, driver, or software can significantly reduce the delivered battery life. In Office Home & Student 2013 RT, we carefully evaluated the impact Office has on the system and then made deep investments to ensure that we met the overall citizenship goals for the platform. For example, ideally Office would have no perceptible user impact at idle. In reality, Office apps need to maintain such things as data freshness which necessarily uses some power. Where we couldn’t remove a feature’s impact on battery, we invested in reducing the extent of that impact. Here are some of the common challenges we addressed.

Software Needs to Let the CPU Sleep

The most actionable thing that drives battery utilization is how often we wake up the CPU to do work, especially when the user is not actively typing, scrolling, etc. CPU power state transitions are expensive. To reduce these transitions, we want to avoid breaking up work across multiple CPU wake ups. Instead, whenever possible, we try to do all the required work at once. We focused on two things to reduce net wake ups: 1) coalesce timers and 2) remove the need for some timers entirely.

The primary mechanism that software uses to ask the system to wake it up is timers. Prior to Windows 8, Windows was not “tickless" meaning you were guaranteed to wake up every ~16 milliseconds. Timers were also often set to wake the process after a specific amount of time or repetitively on a given interval. Windows 8 added new coalescable timers which allow programs to be more battery-life friendly by specifying a range of time to wake up rather than an absolute interval. This allows the system to cluster wake ups with the end result being more CPU sleep time. This extends battery life dramatically.

In Office 2010, there were some situations where applications would wake up the CPU more than 1000 times per minute during idle. With the new Office, we have reduced that by 95% when the user isn’t interacting with the program. Most of the changes are in how Office interacts with the system at a deep level, so you shouldn’t notice any difference. However, there are a few changes that an observant user might notice. One example is the blinking cursor. There is no hardware or operating system support for a blinking cursor so software implements this feature using timers. To minimize the power impact, Office on Windows RT stops blinking the cursor after a few seconds if the user stops interacting with the application. When the user is away, we just show a fixed, non-blinking cursor. This requires no timer and is the best power citizenship option. This is just one example you might notice, and there are several others you won’t notice at all until your battery lasts longer than expected.

Take Advantage of the Hardware

Windows RT is designed to run on ARM System on a Chip (SoC) processors. Unlike a traditional PC where the CPU, graphics card, network adapter, and other systems components are generally all separate pieces, SOC processors combine these components onto a single integrated chip. This improves performance and reduces power consumption. It also guarantees that we can rely on the hardware for expensive operations like playing back videos.

Windows 8 detects this support and takes advantage of it. This is a real benefit to software vendors. For example, in the past, we would write code to detect the level of graphics support delivered by the video card and would write software-based “fallback” code to use when the graphics card in the system did not provide the required capabilities. When using the fallback software-based algorithms, execution cannot be optimized to the extent that can be done when implementing the same algorithm as a dedicated feature of the processor. This impacts both performance and battery life – often dramatically. We’ve taken advantage of these hardware offloads in situations like showing video in PowerPoint. By doing so, we improve the overall battery life of the system.

While the impetus for doing all the work I just described was the desire to deliver great battery life for Office for Windows RT, the fact that it is the same code base as the other Office products means that all users benefit from these changes regardless of what edition of Office they are using.

While running, most programs load or create a great deal of temporary information that is necessary for the program to display information on the screen or to perform other operations. To improve the program’s performance and responsiveness, this information is saved in memory, called caching, so that it can be reused without the overhead of reloading or recalculating. When multiple programs are running the amount of memory available can become limited requiring the operating system to swap out some or all of the program to make that memory available to other programs. This “memory paging” is expensive and can reduce the responsiveness of the system.

Office detects when the user is not actively using the system. When this occurs, Office releases these temporary caches. By having Office determine what memory can most optimally be released rather than relying on operating system memory paging, both Office and the rest of the programs on the system benefit by more effectively utilizing limited memory.

Windows RT systems generally have Solid State Disk (SSD) drives. These drives are fast and battery efficient. However, while the cost per gigabyte continues to decline, these drives are still more expensive that traditional rotational drives. As a result, these drives tend to have smaller capacities. When programs are loaded on today’s large rotational drives that can be over a terabyte in size, most users don’t notice the space used by the program. However, when loaded on a 16GB SSD the operating system and programs can fill a significant portion of the total drive.

While developing Office for Windows RT, we carefully evaluated each feature’s impact on the overall footprint of Office on the drive. For example, rather than including large number of templates and clipart on the drive, we opted to put frequently used templates on the drive while making the more extensive collection available via the start center in Word, Excel, and PowerPoint and also online .

One area that was identified for space savings was language packs. Many OEMs load multiple languages onto the systems they ship. If a user only uses one or two languages, the language resources such as spellers, grammar checkers, and UI strings for the other languages historically have continued to take space on the drive. Office is designed to clean up unused languages packs that are not configured for use by the user. If a user decides later that they want one of the languages that were removed the system will automatically download the required files from Windows Update.

One of the features we expect to see in many Windows RT systems is cellular network support. A characteristic of these networks is that their use is often metered. That is, users pay for the data they utilize – either some amount of money per megabyte of usage or with a monthly cap and then additional fees if they exceed that cap. As a good citizen on these devices we felt we needed to help users know when they are using a metered cellular network and give them the choice whether or not they want to pay for the usage or not take a particular action. Internally we referred to this as helping users avoid bill shock.

Windows RT provides APIs that allow applications including Office to understand the user’s current network state. We can identify if the cellular network is unrestricted or if usage is metered (the user is paying for their usage), if the user is approaching or over their limit, and whether or not they are roaming. When we detect that the cellular network is metered we throttle network traffic to reduce our impact. When users are roaming or over their cap we inform them and give them the options to turn off network traffic. Ultimately, the user knows best what they are trying to do and whether or not they are willing to pay to do it, so we try to give the user more feedback and control.

While the work I’ve just described was done to optimize Office for Windows RT, most users will benefit no matter what edition of Office they choose to use. Office automatically determines whether or not the necessary operating system and device support is present to enable these enhancements. The majority of the changes are enabled when Office is running on any edition of Windows 8. Only features such as support for touch or cellular networks require additional hardware support.

Screen shots of the two applications look very similar because the products are very similar by design. Differences between the versions are subtle. Office Home & Student 2013 RT includes the vast majority of Office Home & Student 2013 features available on PCs, and the features customers use most. Windows RT tablets have special requirements for security, reliability, and battery life, and we’ve worked to ensure that the RT version is well-suited for the platform. Beyond the differences listed below, Office for Windows RT is fully-featured Office with complete document compatibility.

Macros, add-ins, and features that rely on ActiveX controls or 3rd party code such as the PowerPoint Slide Library ActiveX control and Flash Video PlaybackCertain legacy features such as playing older media formats in PowerPoint (upgrade to modern formats and they will play) and editing equations written in Equation Editor 3.0, which was used in older versions of Office (viewing works fine)Certain email sending features, since Windows RT does not support Outlook or other desktop mail applications (opening a mail app, such as the mail app that comes with Windows RT devices, and inserting your Office content works fine)Creating a Data Model in Excel 2013 RT (PivotTables, QueryTables, Pivot Charts work fine)Recording narrations in PowerPoint 2013 RTSearching embedded audio/video files, recording audio/video notes, and importing from an attached scanner with OneNote 2013 RT (inserting audio/video notes or scanned images from another program works fine)

Office Home & Student 2013 RT is only available on Windows RT devices and is not sold standalone. The Windows RT devices available at Windows RT General Availability will include preview editions of Word, Excel, PowerPoint, and OneNote. After the final edition of Office Home & Student 2013 RT is released in a customer’s language, their Windows RT device will be automatically updated with the final edition for free via Windows Update (Wi-Fi connection required). Customers can expect to get these updates starting in early November through January depending on their language. We’ll publish the specific update schedule on October 26 on the Office blog.

Developing Office Home & Student 2013 RT has been a ton of work but also great fun. You’ll find Office for Windows RT to be a full-featured member of the Office family. We hope the work we’ve done helps make Office a great part of your Windows RT experience. We can’t wait to see customers put it to use!

- David Brodsky, Partner Test Manager
- Josh Pollock, Principal Development Manager

.

View the original article here

Want to know how we’re building Windows 8?

Today marks the first post on the Building Windows 8 blog (or “B8” for short). If you’d like to stay in the loop with the behind-the-scenes making-of or engineering of our next Windows release, I suggest checking it out!

View the original article here