FASTforward: Using Search to Achieve a Complete Customer View

February 20, 2008 at 5:01 pm · Filed under FASTforward08

Final session of the conference, and I’m in the financial services breakout track to hear Lee Atkinson, SVP at iDNA, and Marc Hebert, CMO of Virtusa, discuss a real-world example of using enterprise search to build a true 360-degree customer view. They started with some background on the problems with enterprise applications and how dynamic business applications (Forrester’s term for composite applications; he’s quoting from a Forrester report) are starting to take on some of the functionality. Enterprise search applications provide a layer on top of the enterprise applications, files and databases to create search solutions such as corporate search or intelligence solutions. They see enterprise search as a key enabler of dynamic business applications in order to provide both speed and agility of those applications.

The case study that they presented is the implementation of enterprise search in a top ten global bank, providing fast retrieval of structured and unstructured data, and integration with business process management (not a term that I expected to hear at this conference!) to manage events being generated by the system. Their technology platform included enterprise search, BPM, SOA and enterprise portals, and they addressed the 360-degree customer view and a GRC (governance, risk and compliance) event management system. They didn’t use search because of its inherent properties — in fact, the users don’t even see this as a search application, and the applications look like standard structured data extracted from operational systems — they use it to extract information quickly from heterogeneous data sources. Prior to this, the bank had point solutions, multiple data warehouses, and no commonality between systems and databases to provide any sort of consolidated customer view. They needed to integrate data from multiple sources, and also meet their compliance regulations.

They started by providing a compliance and risk view, plus product and customer profiling, in 2005, then expanded to include event-driven business processes and exception management in 2006. In 2007, they brought in the single customer view and more advanced business processes and analysis. Of course, there is no such thing as a single view of a customer in a large organization: different business areas (compliance, operations) have different needs, and each user type’s customized view of a customer is actually a subset of the entire customer model, with other supplementary information pulled in and additional analysis added appropriate to the task at hand. The result is a number of different end-to-end processes, such as know-your-client, anti-money laundering and sanction lists in account opening.

For them, search technology is a way to integrate legacy systems — not an application that I thought of when I considered search — although it requires a deep knowledge of the business domain and the nature of the underlying data. Once the core integration structure has been created, additional data sets and applications can be added quickly and at a fairly low cost. The focus on event-driven business applications based on search results is where search really contributes value to their applications.

Comments

FASTforward: BI goes mainstream

February 20, 2008 at 3:46 pm · Filed under FASTforward08

Philip Bierhoff, Systems Manager at Proctor & Gamble, spoke about strategies to increase user adoption as business intelligence goes mainstream.

P&G’s Symphony project creates "decision cockpits": dashboards based on specific roles and corporate divisions, and including information ranging from traditional BI reports to documents to news.

The underlying data landscape has moved from their first iteration of a common data warehouse in the mid-90s with regional servers plus ETL, storage and aggregation, where BI was driven by stored aggregations; to the current atomic data warehouse with a central server plus ETL and storage, where BI is driven by query rewrite — effectively, aggregation on the fly. They also have SAP generating data into SAP/BW; altogether, they have about 65TB in the data warehouse and 50TB in SAP/BW.

Originally, they used a traditional monolithic BI approach using a BI tool that acted as a portal as well as providing dashboards, graphs and reports as well as its own security layer. They had some challenges with scalability, and the boundary behavior when they pushed the size envelope. They felt restricted by the functionality of the tool, and by the lack of flexibility. Based on these restrictions, they developed a new vision of BI: one based on service-oriented architecture and industry standards, working directly from the corporate data warehouse instead of reporting data marts, and with a thin layer of BI as one of multiple layers of technology that could be changed out as required. The resulting solution uses an SOA-based (I assume JSR168-compliant) portal with various portlets, fed by FAST Radar for BI repots and graphs, and SharePoint for document storage. Under all of this is a service bus connecting to a data services platform, which in turn connects directly to the data warehouses and legacy applications. A WS-Security stack sits alongside to provide standard security and authentication.

They selected FAST Radar because it pairs simplicity with power for fast response times, fits into a portal architecture, downloads to common formats such as Excel and PDF, requires no data model maintenance and can use web services as a data source in additional to traditional database access. They’ve found that it needs very little training for creating and using reports, although they’re still using it only for predefined reports with drill-downs rather than allowing ad hoc report creation. The time to create complex reports has dropped from 30-60 seconds with the old BI tool to 5-10 seconds using FAST Radar.

The resulting architecture looks like a textbook example of how to do SOA correctly. They’re getting a good level of reuse from their services, especially the data access services, and they’re positioned for future extensions without completely overhauling the architecture. They’re able to provide customized dashboards for their 40,000-person user base in a reasonable time frame, in a large part because they’ve designed the underlying services and components for reuse.

Comments

FASTforward: Serving Your Customer in a Real-Time, Multichannel Model

February 20, 2008 at 2:51 pm · Filed under FASTforward08

General sessions are done for the conference, but there’s still some interesting breakout sessions. I’ve moved to the financial services track to hear Ilkka Korkiakoski, a VP at TietoEnator (a European IT services firm) talk about the challenges for the financial services industry as it moves to real-time multichannel customer interaction. Korkiakoski was previously with the OP Bank Group, so he understands this not just from the vendor side, but from the customer side as well; TietoEnator prides themselves on their depth of vertical experience in banking and other specific industries.

He started with a perspective on what’s happening in the financial services industry, particularly in Europe:

• Consumer behavior is fundamentally changing
• The expected response time of a request is now minutes, not days
• The number of customer requests and transactions sent to the back office is 10x that of 1998
• The "Business to Consumer" model will die, and be replaced with "Consumer to Business"
• There has been a shift in strategy within banks from product leadership (1995-2000) to operational excellence (2000-2005) to customer intimacy (2005-now)

The first generation of digital services were created for the digital immigrants, whereas the second generation of digital services are being created for digital natives: those young enough that web technology has always been a part of their life. The first generation included (at the time) innovative — for the time — services such as self service and online access to information and services while still maintaining silos by channel, whereas the second generation will mix business and social components and destroy the barriers between channels for a multichannel consumer experience. To support this, the people within the bank must adapt, and new tools must be created.

Looking at the high-level trends:

• Digital service penetration: basic banking services continue moving to digital channels
• Customer retention: each interaction counts and service experience will be a critical differentiation factor
• Industrialization: growing digitalization and complexity of products and services drive industrialization
• Regulation puts pressure on margins: many initiatives, including SEPA, Basel II, MiFiD, IFRS, Solvency II and SOX
• Industry consolidation: a lot of issues related to integration, economies of scale and differentiation
• Service and product innovations: smaller segments, new types of offerings and bundling will grow rapidly

Moving on to look at opportunities for enterprise search in financial services:

• Customer knowledge, to augment the information in a CRM system
• Sales and customer service desktops, to assist with answering queries
• Service experience in real-time multichannel processes, to supplement the structured data offered to a customer
• Market and business intelligence
• Regulatory and compliance
• Fraud detection, where search and rules combine in complex event processing

Electronic banking has evolved from the basic ATMs of the early 1980s to web banking in the mid 1990s to personalized services in the early 2000s to interactive multichannel services today. As Europe moved to its reliance on internet banking — over 80% of retail transactions are conducted online, with the remainder split between in-branch and the call center — a single platform is required to support the multiple channels, such that the same information and functionality is available regardless of whether the customer is interacting with the bank on the web, in person or over the phone.

The result for the web-based user interaction is a customized portal view that includes supporting information personalized for that customer via contextual search, in addition to the expected structured transactional data.

Comments

FASTforward: Blogger lunch panel

February 20, 2008 at 1:19 pm · Filed under FASTforward08

I sat in on the bloggers’ lunch and the great panel discussion about Enterprise 2.0 adoption challenges, featuring John Hagel, J.P. Rangaswami, Jim McGee and Bill Ives, and moderated by Paula Thornton.

It’s hard to eat and type at the same time, so my notes will be brief, but a few points came shining out of the discussion:

• Resistance to adoption isn’t correlated with age, it’s correlated with position in the company: higher-level people are more resistant to bringing in Enterprise 2.0 technologies because it represents a democratization of content and a relative loss of power at their level. This was a blinding flash of light for me, since it explains a lot about why I, at the age of 47 and hence not the target demographic for most social networking, have so completely embraced social applications, and actively push their use in my customers’ organizations: as an independent consultant/analyst, I have no corporate hierarchy and therefore see the value without a filter of fear.
• The names of social networking applications sound like something from Dr. Seuss, and we all feel a bit silly stating that we blog (as opposed to maintaining a reverse chronological online journal), or use a wiki (as opposed to a collaborative editing workspace). Seriously now, "blog"? "Wiki"? "Mashup"? Do we really expect stuffy enterprise executives to get past the names and see how the technology can impact their organization?
• Adoption relies on familiarity with the technologies and methods of using social applications. Having people immerse themselves in the creation and/or consumption of blogs and wikis in the wild is essential to having them understand why this is important within their company.
• The fact that these technologies are inexpensive (or even free) and quick to implement causes them to be discounted by executives who are used to spending millions on information management systems.

An important but often unexpected effect of Enterprise 2.0 is the emergent uses: users mash up data and functionality to create new content and applications that would never have been imagined by IT or management within an organization.

Comments (1)

FASTforward: The New Polarization

February 20, 2008 at 12:12 pm · Filed under FASTforward08

And now a speaker who we’ve all been waiting for: J.P. Rangaswami, CIO of British Telecom, but likely better known for his blog about information, confused of calcutta. He spoke on the new polarization: how the customer is gaining control, and the impact on corporate IT departments.

We’ve changed from the corporate IT environment of 20 years ago, which was completely permission-based (as in, you had to ask permission to get access to information and applications) and created an arrogant, ivory tower attitude within IT departments. Today, the inmates are gradually taking over the asylum, and the rise in user freedom — and the desire for still more freedom — is sending a chill through traditional IT types. Users are finding ways to work around corporate restrictions to make applications from Skype to Facebook a part of their work life.

Part of a driver for this is the increase in the amount of unstructured information, since many traditional enterprise applications focused almost exclusively on structured information. The first step in dealing with unstructured information in the enterprise came through the spread of desktop applications; the next step, which we’re undergoing now, is focused on dealing with unstructured information on the web. I’m old enough to remember dedicated word processing departments within companies, which centralized control of content creation, and the pain of transitioning to decentralized desktop word processing; someday, we’ll reminisce about the days when the IT department blocked access to Facebook from inside the enterprise.

Another driver for the user revolution that we’re undergoing is the rise of peer-to-peer rather than hub-and-spoke as a model for content creation and distribution. We’re all content authors now. We all recommend content to our friends.

A major hindrance to advancing the user revolution — the polarization that Rangaswami is referring to — is the discounting of youth due to lack of experience, when expertise isn’t actually connected to age in the rise of the technology and communities behind the revolution. A second dimension of the polarization is the democratization of participation, with some people rejecting the idea that good content can be created by amateurs, even in the face of success stories like Wikipedia and the whole open source community. The third dimension of polarization is about time, specifically the speed of implementation, with barriers against the use of products that are proudly still in alpha or beta release, such as Gmail and Google Apps.

The new generation of users — and the new generation of Web 2.0 applications — ignore documentation in the form of manuals, but focuses on self-evident functionality and just knowing what the next step to take.

This genie isn’t going back into the bottle: the power is now in the hands of the consumer, and it’s time to get over the polarization and embrace the user-centered world.

Comments

FASTforward: Search Solutions

February 20, 2008 at 11:44 am · Filed under FASTforward08

Jørn Ellefsen, CEO of Comperio, spoke about customer-driven innovation in search solutions. Comperio creates custom solutions based on the FAST platform, as well as providing a framework that I saw in yesterday’s session on enterprise mashups. Although they work mostly with customers in their Scandinavian home base, they’ve done some significant projects around the world.

This session was done in a semi-panel format moderated by a FAST executive, and Ellefsen was joined partway through by his colleague Stefan Sveen, who I saw in the mashup session yesterday, and and Jan Staff of SPH Search, a local search initiative for Singapore Press Holdings that will provide information and news about Singapore and its people and businesses for locals and visitors. Staff talked about how they were able to focus more energy on the content through the use of the Comperio Front framework, which simplifies access to the FAST platform. Because of the blend of languages used in Singapore, there are some specific search challenges: there are specific English terms, for example, that mean quite different things in Singapore than they do in other English-speaking locations.

Sveen gave a repeat of the demonstration that I saw yesterday, showing how the standard out of the box FAST interface that includes a variety of search navigation/filtering techniques, including the metadata taxonomy and four different "last update" dates. A simple customization using the Comperio Front framework combined the four date filters into a single control, simplifying the user interface. A second, more complex customization dynamically segmented the query search results to show a specialized results tab specifically with food-related results as well as the general search results.

Comperio also provides some productized solutions for less conventional search, such as their Music Search application for searching digital music: it creates a catalog of the digital music assets based on genre, mood and release decade, where search results include 30-second samples and "find similar" capabilities.

Comments

FASTforward: Mobile Services

February 20, 2008 at 11:18 am · Filed under FASTforward08

Timo Teimonen of StarCut, a mobile media publisher, was up to discuss how the mobile internet is transforming. The incredible growth in mobile internet access over the past few years means that people are no longer happy with seeing full-format web pages on their small screen, but want content tailored to the smaller form factor. A huge portion of the market, mostly in Asia, experience the internet only on their mobile, and almost never on a PC except perhaps in a workplace environment. This creates challenges not just for content presentation, but also for search and monetization, with the added issues of localization and contextualization: creating search results and ad content, for example, that’s based on the user’s current location.

The heart of mobile value is based on contextuality: mobile users are willing to trade a bit of their location and preferences privacy in order to get more targeted product and service information. The mobile phone is the bit of information access that people carry with them anywhere, providing the perfect platform for extremely local advertising based on relationships and information about the user. Traditional advertising is extremely inefficient; mobile advertising can be much more efficient at delivering relevant content to people based on where they are and what they’re looking for. In turn, mobile content providers have an incredible amount of information about how people user their content: the Zagat mobile site, which is based on StarCut’s platform, can gather information about what type of restaurant that a customer searches for, be able to provide recommendations based on location and preferences, and even allow for online reservations through the mobile device. In this way, mobile allows advertising and other content to be directly connected to physical retail: a true bridging from the virtual to physical worlds.

I can definitely relate to this, since I live within close proximity of my Blackberry at all time: a device that I use primarily for data access (email, web access, and specialized services such as Google Maps) rather than voice. Mobile content is a big part of my life, and the lives of many of the people with whom I interact, and I believe the quality of this content will greatly improve as localization and contextualization improves.

Comments

FASTforward: Leveraging Search to Drive Innovation in Information Services

February 20, 2008 at 10:17 am · Filed under FASTforward08

Gerry Campbell, president of search and content technologies at Reuters, spoke about how Web 2.0 has set information free from many of the content silos where it used to be entrapped, but there’s a major information overload problem since consumers need related and relevant information presented together in order to provide context. Users want more relevant information, faster; causal relationships between the information and more sophisticated patterns, and the ability to make comparisons.

People don’t just want the "what", they want the "why": where relationships are understood in order to provide insight, not just data. The missing element is context: the layer of meaning created through understanding the relationships of facts and entities, including people, companies, places and events. This layer of contextual intelligence vastly improves the quality of search, and hence information accessibility.

To address this, Reuters is creating a transformational platform. Starting with content and metadata as input — Reuters content, partnered content and the entire web — the information is then processed to understand the content, the queries that will be made against that content, and the needs of the users. The output is not just basic content, but insight and knowledge. ClearForest, a recent acquisition of theirs, will do a lot of that processing to automatically extract and tag content. Furthermore, Reuters is using this to create Calais, an open and free web service to discover entities, events and relations in text and automatically generate metadata, transforming basic text to semantic web content.

Comments

FASTforward: the digital marketplace

February 20, 2008 at 9:56 am · Filed under FASTforward08

Sue Feldman, VP of search and digital marketplace technologies at IDC, spoke about taking the pulse of the digital marketplace. She started by defining the digital marketplace as an online gathering place of buyers and sellers, with particular relationships and dynamics between the types of participants. Users interact with 3 types of sites: gateways (a starting point or index to the web, funded by ad revenue; these are portals and search engines such as Yahoo and Google), hubs (aggregators with high-value tools and trusted content, funded by high-value ad networks because of their trusted position as selectors of content, such as eBay or Amazon), and nodes (communities with few outward connections, such as direct consumer-facing corporate websites).

They’ve created a model of the dynamics of feedback in the digital economy: why people visit certain sites, and why they move from one site to another. Along the way, they found a few surprises:

• The web is fragmenting, not consolidating
• More queries are happening on the web than are reported
• Only a third of queries go through web search engines (and hence are monetized); the rest are directly on destination and speciality sites
• More queries go to Baidu than to Google, Yahoo and Microsoft combined, but all are in Chinese (I have no idea why IDC would find this surprising, except she does admit that prior to hiring an analyst covering China, they had an "embarrassingly North American focus" — what I take to actually mean a US focus)

Looking at the trends that are emerging, they see increasing user sophistication, with users participating in social applications and expecting personalized and localized information. There’s a greater demand for selection on trust, as information overload drives users to speciality sites that pre-select information targeted at their needs. They’re seeing a rise in ad networks that provide context-specific and relevant ads, and a rise in social networks that filter the overwhelming amount of content through a trusted network of friends.

There’s some powerful new technology and infrastructure that’s underlying these trends in the digital marketplace, from personalization and recommendation engines  to user data mining tools. To conduct business on the web, you need to exploit this infrastructure, as well as understanding the long tail effects that impact what you offer to your customers. For traditional businesses, the web is an additional channel, not a replacement for other media.

The dynamics of the digital marketplace are changing quickly, but the roles are becoming defined between the gateways, aggregators and nodes. The central role of the user cannot be overestimated, both on an individual level and as a group. Audiences are attracted not just by useful content, but also by compelling tools and usefulness. Selection and trust are key components for the aggregators and nodes, and targeted communities are replacing generic sites.

Comments

FASTforward: The User Revolution

February 20, 2008 at 9:44 am · Filed under FASTforward08

Safa Rashtchy addressed us in this morning’s first session about the original six trends and impact of the user revolution, from a report that he created while with Piper Jaffray.

1. The emergence of "communitainment": community + communication + entertainment have collided together and are impacting each other’s growth, generating a new type of activity on sites like MySpace. Advertisers must learn to leverage the community aspect of communitainment.
2. The increasing popularity of Usites: sites where user-generated content makes up most of the site, such as YouTube. Time spent on Usites has increased from 3% to 31% of total time spent online, since it’s not just about the content itself, but also about the community that emerges around the content.
3. Mainstreaming of the internet: it’s becoming a mainstream media channel that people are using as part of their daily routine, and 40% of respondents in a recent survey would actually give up TV and keep the internet, given the choice. It has surpassed print media and radio in terms of reach, and is approaching TV. It’s also a key medium in the workplace, further expanding its reach.
4. Declining usage of traditional media: in particular, TV viewing and TV advertising are both dropping.
5. Fragmentation of content consumption across several media: we’ve moved from having only a few choices in how we consume content (e.g., newspapers, broadcast TV, magazines, broadcast radio) to more than 30 different ways, many of them facilitated by the internet. Furthermore, consumers are multi-tasking, merely "snacking" on any particular medium by doing things such as surfing the internet while watching TV.
6. Evolution of user generated brands, especially when well-known corporate brands allow their consumers to impact the corporate brand: consider the Dorito’s Superbowl ad example, where they allowed consumers to design the ad that was aired during the most expensive time available on TV.

Search is the second most commonly used application on the web, and search is becoming the new portal: it’s how people interact with information on the web. Google is increasing their dominance in web search, but local and enterprise search are still wide open. Traditional media is changing their view of search as well: in 2005, Agence France-Presse tried to sue Google because their headlines appeared in Google searches, then by 2007, the Times of London was training its journalists on how to write in a way that would maximize the probability of a high ranking in a Google search.

He offered up nine trends and how they appear before and after the user revolution

• Online versus offline media are changing from being separate and competitive to being integrated into one medium
• Media sources are moving from a few large content providers to vertically-focused multiple sources
• Content control used to be centralized and controlled by publishers, but is becoming fragmented and controlled by the users
• Internet content is changing focus from text-based to video-based; Rashtchy feels that video will be the next killer app on the web
• Main navigation method is changing from portals to search
• Consumer decision process used to be basic and largely influenced by advertising, but is now sophisticated and driven by reviews and rating
• Competitive advantage was based on exclusive content but is now based on simplicity, speed and interconnectivity
• Media consumption patterns
• Social networking

There are four key impacts of the user revolution on enterprises:

1. Enterprise search is not a tool: it’s the platform that will hold your business
2. Consumers expect simplicity, common sense and accuracy; if it’s not easy to use, they won’t use it
3. If you have to use the search constantly, the website is in very bad shape
4. A good search platform must be built on a great website design, created for simplicity, common sense and accuracy

He covered a huge amount of information, and I couldn’t absorb it all; I’d love to hear his talk again.

Comments

FASTforward Day 1 wrapup

February 19, 2008 at 5:43 pm · Filed under FASTforward08

Day 1 at FASTforward has seen some interesting trends emerge.

First and foremost, search is being recognized as not just a feature in other products, but as a profound change in how to we access information. Weinberger’s talk showed just how fundamental those changes are, as data and metadata become one, and how we’re undergoing a user revolution driven not just by user-created content, but by user-generated taxonomies to categorize that content, and user-created links and references to connect the content in new, emergent ways. Clare Hart also discussed the shift in the nature of search and information access through "search without the search box", where information is served up to individuals based on their very customized needs and preferences.

Secondly, search and business intelligence are beginning to at least show a lot of symbiosis, if not actually merging. Tom Davenport’s presentation about competing on analytics showed a trend towards more unstructured data types in analytics, as well as interesting combinations of BI and search. Using search as a basis for BI can increase flexibility by expanding the underlying data sources, and provides a more natural user-driven paradigm for information discovery.

Search has definitely changed how I access information. In many cases, I don’t store information in a predefined structure, but use search — web or enterprise (which for me is an enterprise of one) — to find what I need when I need it. I also have searches buried in automated information presentation, by creating automated blog searches for specific terms that I want to track, and subscribing to them in my feed reader: another form of Hart’s search without the search box.

How is search changing the way that you access information? Do you create your own personalized filtered view of the tsunami of information, or are you intimidated by the sheer volume of what’s out there and not yet comfortable enough with the tools that you need to tame it?

Comments

FASTforward: Enterprise Mashups

February 19, 2008 at 5:14 pm · Filed under FASTforward08

Last breakout session of the day, and it was a tough decision: there was one on business intelligence built on search, but I opted for the enterprise mashups session with Arnt Schoning and Espen Sommerfelt of FAST, plus Stefan Sveen from Comperio, one of their partners that builds search solutions on top of the FAST platform.

They started with some background on service-oriented architecture (SOA) and mashups: SOA as a way to expose business functionality from siloed applications for reusability, and mashups as the Web 2.0 equivalent of SOA, using lighter-weight integration protocols such as REST and JSON instead of WS-*. At last year’s FASTforward, they created a business process in BPEL that consumed a couple of FAST services; this year, they wanted to show a more comprehensive mashup using their technology. There are about 20 services available out of the box with FAST, accessible using Java or SOAP/WSDL.

This turned into a fairly deep dive into the mechanics of how to implement mashups with their services, complete with Java snippets; a higher-level view of the importance of this would have been helpful as at least part of the presentation.

Interestingly, they put what they did last year in the context of purely automated integration-centric processes (hence their mischaracterization of their process as SOA rather than integration-centric BPM), whereas I can imagine that there are also uses of their services as called from processes that include human interaction.

They continued on to talk about mashups: visual versus non-visual, client-side versus server-side, assembly versus coding styles of implementation, and consumer versus enterprise usage. Mashups are ad hoc and situational, hence are primarily user-driven and used to empower users and knowledge workers on the long tail of application development: giving them the ability to create applications that IT would never get around to implementing because the audience is too small or because there’s only a temporary need.

They also talked about the difference between SOA (composite applications) and mashups; they distinguish mashups as being user-created and SOA as being IT-created, but I think that they’re off the mark here: SOA is really just the services layer, which can be consumed by a user-created application just as easily as an IT-created application. The true comparison would be between a composite application development environment and a mashup creation environment, and I think that the line between those is becoming increasingly indistinct.

They showed a demo of a mashup created using their enterprise search services, in which the user can enter any text, and any location information in the free form text is detected and mapped on a Google map. Since they used the location extraction service in their engine to parse the text, words similar to locations are not detected as locations as long as there is sufficient context to make that distinction (e.g., the phrase "Denzel Washington is an actor" did not result in Washington being mapped). Once the points are mapped, other information can be gathered related to those locations, such as weather data or Flickr photos at those locations; this is not really specific to a mashup using FAST services.

A second demo showed the creation of a simple blog with automatic tagging: their automatic tagging service generated the most appropriate tags based on the text of a blog entry. They also showed applying the automatic tagging using a large chunk of text from a real FASTforward blog post; for the second time in a year, I see a recent blog entry of mine being used in a demo during a conference presentation — a slightly odd experience, although they likely had no idea that I was in the audience.

Sveen from Comperio then discussed their best practices in developing search applications: they’ve created a lightweight framework, Comperio Front, that repackages FAST search functionality for easier reuse. The framework is service oriented, and models the search logic workflow and rules as well as encapsulating a number of reusable search tasks and simplifying some of the FAST APIs. It appears to be well-layered, such that (for example) additional adapters could be easily added to the core processing engine to augment the existing web services, HTTP and XML interfaces.

He showed a demo of how they could easily improve the user interface on a standard search portal by combining multiple navigation widgets, and a further demo that shows how search data can be dynamically filtered and displayed. Since I don’t know what the same functionality would look like directly in FAST, I can’t really judge how much easier it is using Comperio Front, although they have the definite benefit of exposing additional search functionality as services for consumption by composite applications.

Comments

FASTforward: Search as Enterprise Portal

February 19, 2008 at 4:09 pm · Filed under FASTforward08

I was delayed getting to the next session on using search as the next generation of enterprise portal by Kara Hansen of Disney, so missed a bit at the beginning. She described how they use FAST to crawl content across multiple enterprise sites to create a virtual view of their content, both enterprise wide and for specific subsites representing divisions within the enterprise. In total, they serve 55,000 employees, plus customers via an externally-facing portal.

One of their most significant challenges was security. First of all, the search crawler needs to have credentials for all of the content locations that it’s going to crawl so as to expose only information that should be exposed on the portal; although these are internal portals, there’s still security issues around content viewing. There’s also an externally-facing portal based on the same technology platform, limited by the specific collections and search profiles to generate a restricted view of the content. When any user executes a search, their security level is combined into the search behind the scenes to deliver only the information to which they should have access. This, in turn, results in different results depending on whether a user is logged in or not, which can frustrate users who don’t understand the implications that their user authentication has on their search breadth.

They also have a challenge with users finding applications that are typically exposed as portlets, but aren’t searchable so don’t end up in the search-based portal. In other words, the user is trying to reach a specific destination on the intranet, not information. To compensate for this, they created a simple XML document that links to the applications and allowed that document to be indexed, although this seems like a bit of a brittle approach since it presumably requires manual updating of that document to add, remove or modify links to applications. However, sometimes a manual workaround can bridge a major gap in functionality; queries for applications now forms a major portion of their executed searches.

They’ve seen a shift in communication culture within the organization: users proactively search for information that they want rather than wait for it to be fed to them. Search also acts as way to introduce user-driven applications, such as wikis and user-created profile pages, a change that Hansen refers to as "profoundly disruptive", in a good way.

They did all of this with a team of five people — three Disney and two from FAST — although that didn’t include the content providers who manage the content that populates each divisional portal.

She ended her presentation by giving away some nice Disney swag: proving that Mickey is for adults too with some MM-themed bar accessories.

Comments (1)

FASTforward: Search Technology in the Institutional Brokerage Industry

February 19, 2008 at 2:55 pm · Filed under FASTforward08

The breakout sessions have started, and I’m sitting in on the financial services track for a couple of sessions, since my primary customer base is in financial services. Adam Sussman at TABB Group is discussing how search technology will change the relationship dynamics of the institutional brokerage industry, where there is a much different relationship between brokers and clients because of the size of the institutional investment portfolios.

The current problem is that even large customers are devaluing the service provided by high-touch (full-service) traders, choosing to make their own trades directly and purchasing research on an ad hoc basis. However, the type of research provided by high-touch firms is really driven by search, providing information about movement in companies that are held within a portfolio, as well as other market conditions that might influence future trades.

More importantly, however, and the focus of Sussman’s talk, is that by implementing sophisticated search technology, the sell side will be able to recognize opportunities to revive high-touch revenues. In other words, use search technology to identify which customers are most likely to be profitable, and focus efforts on those customers.

You can read a bit more about Sussman’s research in this area on their site.

Comments

FASTforward: Next-Generation Innovation in Search

February 19, 2008 at 2:20 pm · Filed under FASTforward08

It’s the first session after lunch, and the first time that we’ve heard an actual FAST presentation, by Bjørn Olstad, their CTO, discussing next-generation innovation in search technology.

They have segmented their audience into two main groups:

• Monetization, which is driving revenue by creating unique user experiences that match customers to relevant assets
• Enterprise, which is driving productivity by creating unique user experiences unlocking actionable information across data silos

Unlike traditional content management systems, search focuses on the intentions of the consumers of the content, not the creators of the content. The search experience should constantly be evolving based on what the consumers are searching for and what they’re doing with the results, plus the constantly changing data underlying the search.

He looked at search as a research portal within an enterprise — which is also covered in a breakout session later today — and other uses that don’t fall into the "search box" view of search. Search is becoming the portal for user interaction, mapping the user intent onto the available content.

We then saw a demo of the new Content Integration Studio, which allows for an easy mapping of reader (data sources) to writer (consumption mechanism): a bit like using Yahoo Pipes to create a graphical mapping from inputs to outputs with various filters and transformations along the way, but using corporate databases and a wide variety of internal and external data sources. You can debug the flow on the fly, setting breakpoints then examining and/or changing data at the breakpoints to see how information will flow through.

They also have some new indexing and contextual matching algorithms in a new search core that allow for near-real-time availability of new content, which is essential for their media/news clients. They’re also providing tools for creating search-based portals to allow search to be the fundamental driver of the user interaction, and provide very customized experiences so that individual users will be presented with different content in their view of the portal.

Comments

FASTforward: Microsoft and FAST

February 19, 2008 at 12:29 pm · Filed under FASTforward08

The morning finished with Jared Spataro, a Microsoft product manager for SharePoint, responsible for enterprise search: namely, the group in which FAST will belong after the acquisition completes. He talked about why Microsoft is making this acquisition, how FAST will benefit, and what this means for customers.

Microsoft started to look at enterprise search about 18 months ago when they realized that their customers no longer saw search as just a feature, but that it was actually shaping the user experience. They found a huge gap in the market between high-end and low-end enterprise search segments, and created a mid-range search capability combined within the SharePoint platform. They also attacked the low end of the market by introducing Search Server Express, a free downloadable product. But that left the high end of the market, which Microsoft is approaching through the acquisition of FAST. So the answer to the question "what’s in it for Microsoft?": total search world domination.

So what’s in it for FAST, besides $1.2B? It gives them scalability both in their ability to innovate and in the massive sales and marketing channel that will expand their product reach. Microsoft has recently bought in to the idea of distributed development, both to find and retain talent and because of non-US acquisitions, meaning that FAST’s development team can stay in Oslo, far from Redmond and hence a bit autonomous.

At the high end, of course, FAST runs on non-Microsoft platforms, and Microsoft is committed to continuing support and innovation across all platforms, a stance that should somewhat calm the fears of existing FAST customers. Spataro also talked about how it’s an advantage to customers that they offer complementary capabilities in enterprise search across the low, medium and high tiers; personally, I’ve never seen a big advantage to having multiple products in the same product class but based on different technologies available from the same vendor, since there will be confusion at the boundaries and some uncomfortable migrations.

He summed up with the overall SharePoint strategy, based on the pillars of business intelligence, collaboration, portal, search, content management and business processes. It’s not their intention to bury FAST inside SharePoint, although it will likely end up as part of the platform at least for branding purposes.

Comments

FASTforward: Competing on Analytics

February 19, 2008 at 12:00 pm · Filed under FASTforward08

Tom Davenport presented on competing on analytics — he published an article with this title in 2006 in HBR. He believes that the planets are aligned for analytics:

• Powerful IT, and a new model for IT use
• Data critical mass: ERP, POS and web data
• Skills sufficiency, at least for the individual pieces
• Business need: a differentiated, personalized, well-informed easy use experience

He makes a distinction between reporting and analytics, and sees analytics as a way for companies to find the best customers and charging them the right price, or analyzing search logs to design a better website.

He sees the following attributes of a user-centric analytical strategy:

• Analytics are used to design and modify the customer experience
• Analytics are used to measure and maximize user engagement
• Analytics are used to determine what the user wants, and personalize to the user
• Analytics are employed across all customer channels
• Analytics are made available to the user

This includes companies such as Amazon and Netflix through their targeted recommendations, Google with customized ads and do-it-yourself analytics, eBay with behavioral targeting, Harrah’s and Marriott with loyalty programs, and Royal Bank of Canada and Capital One with targeted cross-selling across channels. Analytics can also be used for public service: New York City reduced crime through predictive analytics of where crimes were most likely to occur.

He stepped through what’s required for competition grade analytics: data (which he envisions in a data warehouse, counter to Weinberger’s arguments about the more serendipitous discovery of data), enterprise (widespread access to a centralized data store), leadership (executive commitment), targets (focusing analytic activity), and analysts (professional, semi-pro and amateurs, using all levels of analytical tools).

He finished up with the next steps for analytics, from the pursuit of new data types to search/BI combinations to better model management.

Comments

FASTforward: The Information Mess and Why You Should Love It

February 19, 2008 at 11:32 am · Filed under FASTforward08

David Weinberger — author of Everything is Miscellaneous — spoke about the power of digital disorder, and how we need to unlearn what we think that we know about the best ways to organize information. He feels that we’re approaching the end of the age of information — by which he means a focus on rigidly structured information — and a move away from being "informationalized", where we consider everything to be information even if they’re just symbolic representations of reality.

He looked at how many projects, typically physical projects, require a much greater degree of control as they increase in size, but contrasts that with the web, which has growth only because of the lack of control. Control doesn’t scale; we just thought that it did, and managed to scale with control by eliminating information.

There are two orders for how things are traditionally organized: in the case of physical objects, there’s the first order, namely, the physical order in which they are placed; and the second order, which is a catalog of the physical objects, like a card catalog in a library, which uses a limited and pre-determined taxonomy for categorizing the objects.

In the new world of organization, we allow for user-generated folksonomies to generate metadata that we couldn’t have possibly thought of for the objects. These multiple layers of metadata online allow for a physical object to virtually be in more than one place at a time; when the concepts are applied to online data rather than physical objects, the division between data and metadata starts to blur. All contents are also connections: everything leads to everything else, creating a wonderfully messy mass of interconnected data. The web, of course, excels at creating connections because of the basic premise of linking: we create hypertext links on pages to make connections that are important to us. The user revolution, therefore, is not just about us creating our own content; we also control the links, hence control the connections between content and the organization of that content. Digg, Twitter, your RSS feeds and other socially-created sites create our new "front page", replacing the newspaper of old: why would you read someone else’s idea of what’s important, rather than self-select what you’re interested in reading?

A key thing is not to set arbitrary limits; he cites the example of the Library of Congress recently posting a huge set of historical photos on Flickr to allow open tagging by everyone, which has since run up against Flickr’s arbitrary limit of 75 tags per photo. Tagging like this helps to destroy stereotypes: the more data/metadata that you have about someone or something, the less likely you are to make a shallow characterization.

Weinberger’s a great speaker, and uses some really funny and compelling images in his slides.

Comments (1)

FASTforward: New Patterns for Interacting with Information

February 19, 2008 at 10:00 am · Filed under FASTforward08

Clare Hart, EVP of Dow Jones‘ Enterprise Media Group spoke about rising expectations and new patterns for interacting with information. She started with the four macro trends that are impacting business and consumers today:

• Globalization
• Distributed workforce
• Follow the sun business cycle
• Information overload

Information is being exchanged around the world instantaneously, and this constantly flowing river of information is changing how companies react to events and fundamentally how they do business. Information overload — actually relevancy overload — makes it difficult to find the right information unless there are tools such as sophisticated taxonomies and tagging applied to properly filter the information. In my view, lack of relevancy is the most common reason that people ignore information that is available to them: some people don’t Google for information because they have a difficult time separating the wheat from the chaff.

A couple of years ago, Dow Jones acquired Factiva, which provides search and filtering of external news information to provide customized feeds to their customers; she stepped us through how their Factiva service presents this information in order to provide the most relevant information from traditional news sources as well as blogs and other non-traditional news sources.

She had an interesting cause-and-effect slide on how an event occurs, that triggers news, commentary and analysis that feeds into a discussion on that event and its analysis. For example, an investment banker interested in a specific industry would be presented with a screen of information in the event of a significant loss of a company in that industry: the original news story, internal and external research, market reactions and other related information that makes up the bigger picture driving potential investment in that industry. From this, he can drill through to more detailed contextual information on the specific company and its competitors, as well as general market information. For a different role, such as a wealth manager managing a client portfolio, the same information about a company’s losses has different meaning, hence is presented in a different way: exposure to that company within a client’s portfolio as well as specific client information and even news articles related to the client’s investment strategy and personal interests.

As she points out, this is search without the search box: the search is executed behind the scenes and the results pushed through to the recipient based on his known role and interests. The investment banker is looking to improve his investment-related reactions to the event; the wealth manager is looking to improve the relationship with his client.

She finished up with the benefits of search without the search box:

• Anticipatory discovery
• Contextual linking
• 360 degree view
• understand business from multiple angles
• Drive confidence
• Ability to act

Comments

FASTforward: the user revolution

February 19, 2008 at 9:30 am · Filed under FASTforward08

The wifi continues to be flaky — good connection strength but no actual connectivity — but there’s power at the tables which almost makes up for it. Almost.

We started the day with John Markus Lervik (plus a light show, an enormous inflated ball, a small bit of theater and a live band) discussing the theme of the conference: the user revolution. There’s a strong relationship emerging between business intelligence and search, and the judicious use of enterprise search has moved beyond a commodity to become a competitive differentiator as well as playing a critical role in many government programs (the ones that keep tabs on all of us). Google made us all see that we can use search for finding information and relationships that were never available to us before; enterprise search such as FAST gives that same ability to employees inside an organization, allowing them to search the corporate information assets in order to do their jobs better. No longer are users stuck with a static view of information created by a report designer, but can create their own views for their own purposes.

John Hagel of the Deloitte & Touche Silicon Valley Research Lab spoke to us next about the impact of the user revolution on your organization: how it’s putting pressure on traditional organizations because of reduced product life cycles and other factors, contributing to life spans of companies.

He spoke about the forces that are driving the user revolution within business:

• The move from the scarce resource of limited shelf space for products and services to unlimited shelf space: what drives the long tail. The scarce resource is now our attention. There’s a much lower cost of interaction now, including finding information about vendors and switching to a new vendor. Customers are gaining more power.
• There’s a shift to intangible assets — mostly human talent — as key competitive differentiators rather than the old model of competition based on physical assets such as plants. There’s greater competition for this talent, since it’s much easier for people to move to another organization or even go out on their own; the key is to become a magnet for attracting talent by creating the right sort of environment, particularly an environment that builds talent. Talent is gaining more power, too.

He went on to discuss the change from push to pull business interaction models, and how there’s a move to some new performance metrics: return on attention (e.g., measuring profitability by customer rather than by product), return on information (e.g., what can we find out about participants/customers by their interactions with us, and how can we provide them with better information) and return on skills.

Hagel made a big deal at the beginning of his presentation about how he wasn’t using a PowerPoint deck, but then proceeded to read his presentation from notes in a mostly uninflected voice, making it difficult to stay focused on his talk — I’m sure that I missed a lot here. I never thought that I’d say this, but his presentation might have actually been improved by the use of PowerPoint.

Comments (2)