Semantic Search Engine

Back to the days in the university, semantic web was a computer science research area and now we can see some nice search applications out there: Powerset / hakia / deepdyve

Compare the results of searching"what is software design" in Powerset , hakia and Google

What is semantic search? Wikipedia says, "Semantic search is a process used to improve online searching by using data from semantic networks to disambiguate queries and web text in order to generate more relevant results. (more)"

Here's a very good summary from Atos Consulting:

Concept

* The Semantic Search Engine or Third-Generation Search Engine is a search engine in which Semantic technologies and / or Natural Language Processing (NLP) techniques are applied. There are some investigation groups and companies―such as Swoogle, PowerSet (acquired by Microsoft in 2008), and hakia ― that are making a big effort in this area.
* Basically, the objective of these products is to take advantage of the existence of structural data (RDF, microformats, and OWL) on the Web in order to process the information included and offer to the end user an intelligent response. Essentially, the Semantic Search Engine can understand the question and is able to provide an exact response, rather than just returning thousands of pages where all words included into the search field are found.
* This challenge is so complex that most current providers cannot properly be considered as true Semantic Search because results are not oriented to the casual end user, but as a tool for specialists.
* Another related concept is Natural Language Search in which one can ask a question using complete sentences phrased in everyday language (such as ‘where can I find information about William Shakespeare?’) as opposed to formulating a search statement (such as ‘su:Shakespeare, William’).
* There is a big dependence between this technology and Semantic annotation. Until all Webmasters, blog users, and wiki writers frequently use ontology annotation, Semantic Search will be limited to certain Webpages or content provided by Semantic experts.
* Another important dependence to consider is with the language itself. Some languages (such as English) are more suitable than others in making the adoption process less complex.

Application and Benefits

* Different applications of Semantic Search are: Vertical Semantic Search and General-purpose Semantic Search. General-purpose actual search engines work using statistical ranking methods, and the results are reasonably acceptable, although the end user’s time may be wasted by search results that don’t distinguish between credible contents and misleading information. As hakia points out, a popular Website may not always be credible and a credible Website may not always be popular.
* Another application would be the ability to search precisely in non-textual content, such as annotated photograph and audio.
* Related with Vertical Semantic Search, this technology is being adopted where the meaning of sentences, the combination of content, and context are crucial to retrieve precise information, such as in law, finance, science, and libraries.

SharePoint Video on YouTube

Get Started with SharePoint

SharePoint 2010 updated services infrastructure

In Microsoft® SharePoint® Server 2010, services are no longer contained within a Shared Services Provider (SSP). Instead, the infrastructure for hosting services moves into SharePoint® Foundation 2010 and the configuration of service offerings is much more flexible. Individual services can be configured independently and third-party companies can add services to the platform.

More granular configuration of services
· You can deploy only the services that are needed to a farm. Services that are deployed are called service applications.
· Web applications can be configured to use only the services that are needed, rather than the entire set of services that are deployed.

· You can deploy multiple instances of the same service in a farm and assign unique names to the resulting service applications.
· You can share services across multiple Web applications within the same farm.

Sharing services across farms
· Some services can be shared across server farms. Other services can be shared only within a single server farm. Services that support sharing across farms can be run in a central farm and consumed from regional locations.
· Each Web application can be configured to use services from different farms. For example, you can share the User Profile Service across Web applications in several server farms while using some services, such as the Business Data Connectivity, locally.
· In large environments, computing-intensive services can be run in a central farm to minimize administration overhead and to scale out easily and efficiently as requirements grow.

Logical architecture
· All service applications in a farm are deployed to the same IIS Web site. (See below diagram)
· Service applications can be deployed to different application pools to achieve process isolation. However, farm performance is optimized if services are contained within one application pool.
· To achieve physical isolation of services, create separate instances of service applications and place them in different application pools

Service groups
· By default, all service applications are included in the default group, unless you change this setting for a service application when it is created. You can add and remove service applications from the default group at any time. (See above diagram)
· When you create a Web application, you can select the default group or you can create a custom group of services. You create a custom group of services by selecting only the service applications that you want the Web application to use.
· Custom groups are not reusable across multiple Web applications. Each time you select "custom" when creating a Web application, you are selecting services only for the Web application you are creating.

Connecting service applications to Web applications
· When you create a service application, a connection for the service application is created at the same time. A connection is a virtual entity that connects Web applications to service applications.
· In Windows PowerShell these connections are called proxies. The term "proxy" also appears at the end of the type description for connections on the Manage Service Applications page in Central Administration.
· Connections for services in the local farm are not created by the administrator, but these appear along with the list of service applications in Central Administration.
· Some connections might include settings that can be modified. For example, if a Web application is connected to multiple instances of the Managed Metadata service, you must indicate which of the connections is connected to the primary service application which hosts the corporate taxonomy.


A List of Service in SharePoint 2010

Sample of Single farm, single service group

Manage Service Applications page

What is Windows Communication Foundation?

I've been designing .NET applications using ASP.NET Web Services since 2004. Web Services is the great tool for building service-oriented architecture (SOA) systems and connecting applications in different environments. But, it does not solve some other important problems and you need to use another tools e.g. .NET Remoting, Enterprise Services, Web Services Enhancements (WSE), System.Messaging, System.Net.

With WCF, the perfect solution is at hand:



Here’s how WCF addresses common technical requirements:
-  Because WCF can communicate using Web services, interoperability with other platforms that also support SOAP, such as Java EE application servers, is straightforward.
-  To allow optimal performance when both parties in a communication are built on WCF, the wire encoding used in this case is an optimized binary version of SOAP. Messages still conform to the data structure of a SOAP message, referred to as its Infoset, but their encoding uses a binary representation of that Infoset rather than the standard angle-brackets-and-text format of XML.
-  Managing object lifetimes, defining distributed transactions, and other aspects of Enterprise Services are now provided by WCF. They are available to any WCF-based application, which means that third  party applications can use them with any of the other applications with which it communicates.
-  Because it supports a large set of the WS-* specifications, WCF helps provide reliability, security, and transactions when communicating with any platform that also supports these specifications.
-  WCF’s option for queued messaging, built on MSMQ, allows applications to use persistent queuing without needing to use another set of application programming interfaces.
-  The version of WCF in the .NET Framework 3.5 has built-in support for creating RESTful clients and services.

Read more here: Introducing Windows Communication Foundation
Or MSDN WCF

ADO.NET Entity Framework (Object/Relational Mapping)

You may heard of Hibernate if people are talking about ORM. What ORM tools can Microsoft deliver? Here you go:

The ADO.NET Entity Framework is an Object/Relational Mapping (ORM) framework that enables developers to work with relational data as domain-specific objects, eliminating the need for most of the data access plumbing code that developers usually need to write. Using the Entity Framework, developers issue queries using LINQ, then retrieve and manipulate data as strongly typed objects. The Entity Framework’s ORM implementation provides services like change tracking, identity resolution, lazy loading, and query translation so that developers can focus on the application-specific business logic rather than the data access fundamentals.




The Entity Framework uses the Entity Data Model (EDM) to describe the application-specific object or “conceptual” model against which the developer programs. The EDM builds on the widely known Entity Relationship model (introduced by Dr. Peter Chen) to raise the abstraction level above logical database schemas. The EDM was developed with the primary goal of becoming the common data model across a suite of developer and server technologies from Microsoft.

Extract from ADO.NET Entity Framework At-a-Glance
Read more on Beginner's Guide to the ADO.NET Entity Framework

Are you really know how to program in .NET?

Have been interviewing many .NET developers/programmers/analysts recently, to be honest, many of them are below average. I've consolidated some essential .NET development materials and resources on the web for those who keen on learning software development:

Overview of the .NET Framework : Entry ticket please. Framework architecture, Common Language Runtime, Common Type System, Class Library, Guidelines and Best Practices ...

C# Programming Guide & C# Language Specification 3.0 : Arrays, Strings, seems easy? Not quite. Generics, LINQ Query Expressions, Lambda, Iterators, Collection, Threading, Reflection ...

Design Guidelines for Class Library Developers : Good codes, bad codes. That makes a difference.

ASP.NET and Visual Web Developer : How to build an advance ASP.NET web site. What's more? Web Services, AJAX, Model View Controller (MVC)

Improving .NET Application Performance and Scalability : The need of speed.

Microsoft Data Development Technologies At-a-Glance : At the end of the day, data is what you really concern. So, how to access data is critical.

Microsoft Application Architecture Guide : For developers or solution architects to design and build effective, high quality applications. It provides guidance for using architecture principles, design principles, and patterns that are tried and trusted.

How to measure your web site?

Measuring web site traffic and marketing effectiveness is hard and often ignore during software development. Some web analytics software provide extensible solution with a consolidated view of visitor interactions. Omniture and Google Analytics are the enterprise-class web analytics solution that gives you rich insights into your website.

Quote: "Free product offerings are tempting, and a good learning tool, but strategic initiatives with an opportunity for large gains in site yield should choose commercial products." "Narrow a shortlist of Web analytics vendor choices to those that answer the big question, which is software as a service (SaaS) or on premises, then concentrate on price, support and "ecosystem" partners."



Other players: Coremetrics, Unica, Webtrends

Read more:
The Interactive Marketing Metrics You Need - Forrester Research
Key Challenges in Web Analytics, 2009 (PDF)

Flash vs Silverlight

Still enterprises are still prefer Flash than Silverlight at the moment. But Silverlight has been greatly improved in these few years. Have a look:


Flash vs Silverlight Gallery
(you will be more interested in this comparison)

Silverlight Showcase

Silverlike

Designing a Knowledge Management System

In the past four years, I've managed to implement for about 3 Knowledge Management Systems in 3 large listed enterprises, one is a railway company, one is a sport company and the other is an international sourcing and retail company. I want to share my view on the succeed factors of building a KM system in a real world:

• Sell it as a corporate-wide initiative and secure funding and supports from top management
• Select a group of KM champions in every business unit
• Choose a right software platform
• Design a simple document taxonomy
• Build a community of practice
• Connecting people using web 2.0 technologies
• Keep the content of KM portal flash and up-to-date

A knowledge handbook of a great KM project in Hong Kong.

Critical-Thinking Questions [1]

• Why would a company need a system for managing knowledge?
• How is knowledge managed at your organization? Is there a centralized repository in place, or do you engage in impromptu sharing?
• How would you organize your project documentation, so that it is convenient for you?
• How do you see your company's business processes changing, if you implemented a knowledge-management system?
• For a knowledge-management system to be successful, the technical architect should select the KMS based on what criteria?
• What is the ROI of the knowledge-management system? How would you go about measuring it?

Articles on KM:
Architecting a Knowledge-Management System [1]
Six ways to make Web 2.0 work

Model View Controller (MVC)

I had a long discussion with a friend of mine on applying Model View Controller (MVC) in modern web application development. My view to the value of this design pattern is: It depends!

You can read more about MVC on ASP.NET MVC , MSDN, Java Patterns or a pet shop example.

.

SharePoint development patterns and practices

The Developing SharePoint Applications guidance helps architects and developers design and build applications that are both flexible and scaleable.
http://msdn.microsoft.com/en-us/library/dd203468.aspx

The TechNet best practice resource centre is another good resource.
http://technet.microsoft.com/en-us/office/sharepointserver/bb736746.aspx

Or checkout my SharePoint reading list:
http://delicious.com/paulabraham/sharepoint

.

The Practice of Design

Our customer says: “I want our new system to have a response time of no more than two seconds!!”
Our IT Director says: “We must cut cost!! All new development efforts must fit within our existing environment, using existing tools.”

We have faced these challenges many times and are used to design challenges that have both stringent customer requirements and built-in IT constraints due to existing environment or technology realities.
So what do you do in such cases? You dig deep and become very creative. You pull out all your design skills. What does that mean? If you are like us, you have a solid foundation in design methodology, you have strong technical skills, and you have wellhoned ‘people skills’ like negotiating and communicating. Then, you put them all together and get to work!

As Architects, we have found that in order to meet the challenges placed on us by both the customer and our Information Technology leadership, we must understand the whole system of interconnected elements that participate in, impact, and influence the design process. The scope of a software architect’s world is awesome, as the breadth of experience documented in this book demonstrates...

Extract from ITABOK.

.

Becoming an Architect in a System Integrator

So inspired by Amit Unde after reading this article.

...

Being an architect is tough! What architects do is a mystery to much of the world; this is hardly surprising, because an architect's work is intangible—"thought-ware," if you will—and it happens in the background. That makes many wonder about the architect's role in an organization. Architects interact with many stakeholders—CIOs, project managers, business users, and developers—and each expects them to work differently. While the CIO expects an architect to derive a solution road map for implementing the company's IT vision, the developer expects the architect to provide direction on the technical problem. The architect needs to have a bird's-eye view in one scenario, while in some other scenarios, the architect needs to dive deep into the problem area. The architect is expected to be both a generalist and a specialist.

Many companies try to reduce the ambiguity by introducing different flavors of the role, such as enterprise architect or solution architect. Ironically, differentiation within the role can add to the confusion since there is no standardization of the designations across companies. Let's find the commonalities and define these different flavors of the role.

Read more here: http://msdn.microsoft.com/en-us/architecture/cc505970.aspx

.