0

SP Error: ‘New Document’ requires a Microsoft SharePoint Foundation-compatible application and web browser

We encountered the following error when ever a user tries to create a new document from any SharePoint document library using Internet Explorer (IE). It works fine in all other browsers.

Note: We are using On-Premise SharePoint 2013 and Office 2010 Professional plus.

“New Document requires a Microsoft SharePoint Foundation-Compatible application and web browser. To add a document to this document library, click the ‘Upload Document’ button”

IE pop up for new word documents

IE pop up for new word documents

Based on our observation it is due to the missing add-ons,  that get added to IE along with the Office package.
I believe some updates to the office removed these or the Office installation got corrupted due to patch updates or something.

SharePoint Add-ons

SharePoint Add-ons

Solution: Repair your office installation.

  1. From the Start menu, click Control Panel, and then click Programs.
  2. On the Control Panel, click Programs and Features.
  3. Scroll down to the list of installed programs, select Microsoft Office, and then click ‘Change’.
  4. In the Change dialog, click Repair and then click Continue.

Conclusion:

Hope this post helped to resolve your issue. If it did not, all the best in troubleshooting the root cause and please leave a comment on how you resolved it. Happy coding!

0

An error occurred accessing your Microsoft SharePoint Foundation site files

We encountered the following exception while trying to open a SharePoint 2013 site using SharePoint Designer

“An error occurred accessing your Microsoft SharePoint Foundation site files. Authors- if authoring against a web server, please contact the Webmaster for this server’s Web site. WebMasters – please see the server’s application event log for more details.”

SharePoint Designer Error

SharePoint Designer Error

After exhaustive troubleshooting, we discovered that New Relic configuration is what was causing the issue. This might or might not be the case for you!!

Step 1: You could first try to disable the New Relic agent altogether  and see if that is causing the issue.

Following is the screen capture to show the path to navigate and the XML Element to change to  disable the .NET Agent for New Relic, change agentEnabled = “false”

New Relic Agent Enabled

New Relic Agent Enabled

Once you have confirmed it’s the New Relic  that’s causing the issue, try the following option.

Step 2: Disabling the Browser Auto Injection with Agent still enabled.

The following screen capture clearly indicates the changes that are needed to disable the Browser Auto Injection with Agent STILL ENABLED.

Browser Auto Injection disabled

Browser Auto Injection disabled

Browser Auto Injection automatically injects a piece of Javascript code to the <head> of the page that tracks front end page views and other end user metrics. Here is our documentation on this: https://docs.newrelic.com/docs/browser/new-relic-browser/additional-standard-features/page-views-understanding-your-sites-popularity

The impact of disabling Auto Injection is that you will no longer be receiving these end-user metrics unless you use the copy/paste method to manually add the Javascript code, described here: https://docs.newrelic.com/docs/browser/new-relic-browser/installation-configuration/adding-apps-new-relic-browser#copy-paste-app

There are a few options for adding the Javascript in Sharepoint, you could:

  • Create a new master-page, and add the Javascript to the <head> there. https://msdn.microsoft.com/en-us/library/dn205273.aspx
  • Use the AdditionalPageHead delegate control in your SP solution. http://www.fivenumber.com/understanding-sharepoint-delegate-control/
  • Using Designer, add the script as you would in any other HTML editor. However, we’ve witnessed odd behavior where Designer will place the <head> tags inside the <body> tags, which has caused copy/paste instrumentation not to work. The solution here is to add the script right below the DOCTYPE declaration, outside of any actual tags. The script will still be rendered inside of <head> tags on actual deployment, though.

Conclusion:

The idea behind sharing this information is to encourage you to look at other dependencies like network configuration, third party modules or agents etc. that might be causing the issue. In our case it was New Relic, it could be something else for you.

Disclaimer: New Relic is one of THE BEST monitoring tools available in the market. Undoubtedly! It is just that you need to know how to tweak the configuration tailored to your needs.

0

Using JavaScript Promises with SharePoint 2013 – Deferring executeQueryAsync to be a little Synchronous

If you have worked with Asynchronous programming, its no brainier that you might have enjoyed writing highly performing apps.

But I am sure you might have felt at some rare occasions that “Alas! I wish if this small portion of my code works synchronously and dammit wait for an action to complete!”. 

Most of the time its always fire-and-forget scenario with asynchronous calls, but there are certain scenarios where you want your code to return something. Especially this gets even tricky if you are working with asynchronous logic within loops. JavaScript Promise comes to your rescue!

This blog post is not meant to educate you on JQuery or JavaScript Promises, but just to give you an idea of something like this exists and how you can use it with SharePoint. I will not leave you disappointed, following are some good articles that will get you started.

Article 1: http://blog.qumsieh.ca/2013/10/31/using-jquery-promises-deferreds-with-sharepoint-2013-jsom/

Article 2: http://www.vasanthk.com/jquery-promises-and-deferred-objects/

Spread the word for your fellow developers

“JavaScript Promise 

I was working on a typical use-case of updating SharePoint list items for a SharePoint Online site. Let me explain the use case in detail, I have two SharePoint lists, one is a SharePoint Document Library and the other is a Custom list. Document library has to be updated based on certain values from the Custom list. Of course, the data has to be manipulated based on certain conditions and multiple business rules before updating the document library.

Initial thought process would be to use any of the tools at disposal or use out-of-the-box features to get this working, but after an exhaustive research over the internet, I could not find any way to accomplish this with no customization. So as a last resort, I decided to write some code.

NOTE: This is SharePoint online site, REST API or JSOM (Java Script Object Model) would be the default choices. I am using JavaScript for this example.

Following is the general thought process

Step 1: Get the list of items asynchronously  from Source List (i.e. SharePoint Custom List)

Step 2: Loop through the items and update the Destination List (i.e. SharePoint Document Library)

Ideally everything should work seamlessly, but Step 2 will fail with asynchronous way of programming. When you loop through the items that are fetched from Source List it works great, but any updates or actions that you perform with in a loop may not work.

Loop proceeds irrespective of action being complete or not. If you are dealing with one item and no-loop involved it works fine. But I am dealing with hundreds of items with too many of these loops.

Using JQuery Promise for SharePoint 2013

 

NOTE:

Following  code has clear and detailed inline comments. Everything should be self explanatory!

Following code has been developed for one-time use only and is meant for educational purposes only and not recommended for production use. I can say upfront that the naming conventions, error handling/logging are not production ready. You may need to make updates to the code and may require some re-work. Of course, there is always a better way doing things.Use at your own risk!


<!--Add JQuery reference-->
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.12.0/jquery.min.js"></script>

<script type="text/javascript">

//Get the source list items to loop through and update the destinantion list.
function GetSourceListItems() {

//Assign the name of the list to use as Source List
var strSourceListName = "SourceListName";

//Get the current SharePoint Context: Represents the context for objects and operations
var oContext = SP.ClientContext.get_current();

//Get the current web
var oWeb = oContext.get_web();

//Get the source list using the Name or GUID
var oList = oWeb.get_lists().getByTitle(strSourceListName);

//CAML Query to query the Source List
//Following is the Query to get items less than ID 800.
var strQuery = '<View><Query> <Where> <Leq> <FieldRef Name=\'ID\' \/> <Value Type=\'Counter\'>800<\/Value> <\/Leq> <\/Where> <\/Query><\/View>';
var oQuery = new SP.CamlQuery();
oQuery.set_viewXml(strQuery);

//Get the items based on Query
var allItems = oList.getItems(oQuery);
oContext.load(allItems);

//Make an asynchronous call.
oContext.executeQueryAsync(Function.createDelegate(this, function () { onQuerySuccess(allItems); }),
Function.createDelegate(this, this.onQueryFailed));
}

//On Query success loop through the items and update the destination list:
function onQuerySuccess(allItems) {

//Returns an enumerator that iterates through the SP.ClientObjectCollection Class.
var ListEnumerator = allItems.getEnumerator();

//Names of the columns in the source list to use
var sourceColumnArray = ["SourceListColumn1", "SourceListColumn2", "SourceListColumn3"];

//Iterate though list items and update the destination list.
while (ListEnumerator.moveNext()) {
//Get the current item
var currentItem = ListEnumerator.get_current();

//Create an array of values that should be used to update the destination list.
var valuesArray = [];

//Loop through the source column array and populate the array.
for (var i = 0; i < sourceColumnArray.length; i++) {
var value = currentItem.get_item(sourceColumnArray[i]);
valuesArray.push(value);
}

//Get the value of the common key that relates both the Source and Destination Lists.
//Use the same key to query against the destination list.
var commonKey = currentItem.get_item('commonColumnName');

//Check the length of array and send it update the destination list.
if (valuesArray.length > 0) {
UpdateDestinationListItem(commonKey, valuesArray);
}
}
}

//On Query failure log message
function onQueryFailed(sender, args) {
//Log Error
}

//Update the destination list
//Use common key to query the distination list
//Use values array to update the values of the desination list
function UpdateDestinationListItem(commonKey, valuesArray) {
//****Important 1.0****//
//Use of JQuery deferred
var d = $.Deferred();

//Give the name of the distination list to update with values.
var destinationListName = "DestinationListName";

//Use the key from the source list to query the destination list.
var strDistinationCAMLQuery = '<View><Query><Where><Eq><FieldRef Name=\'FileLeafRef\' \/> <Value Type=\'File\'>' + commonKey + '<\/Value> <\/Eq> <\/Where> <\/Query><\/View>';

//Get the current SharePoint Context: Represents the context for objects and operations
var oClientContext = SP.ClientContext.get_current();

//Get the current web
var oWeb = oClientContext.get_web();

//Get the destination list using the Name or GUID
var oDestinationlist = oWeb.get_lists().getByTitle(destinationListName);

//Specifies a Collaborative Application Markup Language (CAML) query on a list or joined lists.
var oCamlQuery = new SP.CamlQuery();
oCamlQuery.set_viewXml(strDistinationCAMLQuery);

//Get the list of items using the CAML Query
var oCollListItem = oDestinationlist.getItems(oCamlQuery);
oClientContext.load(oCollListItem);

//****Important 2.0****//
//Use of JQuery deferred
//Send list of items, client context, array of values from Source List
var o = { d: d, listOfItems: collListItem, clientContext: clientContext, valuesArray: valuesArray };

clientContext.executeQueryAsync(Function.createDelegate(o, successCallback), Function.createDelegate(o, failCallback));

return d.promise();

function successCallback() {

var itemCount = this.listOfItems.get_count();

var destinationColumnArray = ["DestinationListColumn1", "DestinationListColumn2", "DestinationListColumn3"];

var listItemInfo;

if (itemCount > 0) {
//****Important 3.0****//
//Use of JQuery deferred: Using 'listOfItems'
var listItemEnumerator = this.listOfItems.getEnumerator();

//Iterate though the list of items fetched from destination list
while (listItemEnumerator.moveNext()) {

var oCurrentListItem = listItemEnumerator.get_current();

for (var i = 0; i < destinationColumnArray.length; i++) {
//Write to console for debugging
console.log('Updating Value');

//****Important****//
//Destination list item getting updated.
oCurrentListItem.set_item(destinationColumnArray[i], this.valuesArray[i]);

//Write to console for debugging
console.log('Update Successfull');
}
oCurrentListItem.update();

this.clientContext.executeQueryAsync(onUpdateItemSuccess, onUpdateItemFailed);
}
}
}

function failCallback() {
this.d.reject("Failed at failCallback() method");
}

function onUpdateItemSuccess() {
console.log("Item Updated Sucessful");
}

function onUpdateItemFailed() {
console.log("Item Updated Failed");
}
}
</script>

I am not sure if this blog post helped you with what you wanted, but I would be glad if  you have discovered about JavaScript Promises for the first time via my blog post. Happy Coding!

0

Introduction to PowerShell for SharePoint 2013 -1 (Developer)


<#
.SYNOPSIS
--------------------------------------------------------------------------------
Code generated by: Susheel Dakoju., Windows Powershell Host v 3.0
Generated on: 04-11-2015 19:28
Generated by:
Organization:
--------------------------------------------------------------------------------
.DESCRIPTION
- This script is generated for Capital Area .NET SharePoint Special Interest Group Meetup demonstration of basic Powershell commands for SharePoint
- This script is running against SharePoint 2013 powered by Powershell version 3.0
- Run as administrator to run the script sucessfully.
- This script is not for production use. The commenting and Intending is done to suite for quick demonstration and may not be adopted for organizational use
--------------------------------------------------------------------------------
#>

#---------------------------------------------------------------------------------------#
# Getting Started
#---------------------------------------------------------------------------------------#

#---------------------------------------------------------------------#
#1. Get Current Powershell Version
#---------------------------------------------------------------------#
$PSVersionTable.PSVersion

#---------------------------------------------------------------------#
#2. Get count of Commands with 'SP' in the name SharePoint Snap-In
#---------------------------------------------------------------------#
Get-Help *sp* | Measure-Object

#---------------------------------------------------------------------#
#3. Get count of Commands with 'SP' in the name BEFORE ADDING SharePoint Snap-In
#---------------------------------------------------------------------#
Add-PSSnapin -Name Microsoft.SharePoint.Powershell
Get-Help *sp* | Measure-Object

#---------------------------------------------------------------------#
#4. Get count of Cmdlets with 'sp' in the name AFTER ADDING SharePoint Snap-In
#---------------------------------------------------------------------#
Get-Command *sp* | Where {$_.CommandType -match "cmdlet" }| Measure-Object
Get-Help *sp* | Measure-Object

#---------------------------------------------------------------------#
#5. Get count of Applications with 'sp' in the name AFTER ADDING SharePoint Snap-In
#---------------------------------------------------------------------#
Get-Command *sp* | Where {$_.CommandType -match "Application" }| Measure-Object
Get-Help *sp* | Measure-Object

#---------------------------------------------------------------------#
#6. Get count of Applications with 'sp' in the name AFTER ADDING SharePoint Snap-In
#---------------------------------------------------------------------#
Get-Command *sp* | Where {$_.CommandType -Match "cmdlet" -and $_.ModuleName -Match "Microsoft.SharePoint.PowerShell" }| Measure-Object
Get-Help *sp* | Measure-Object

#---------------------------------------------------------------------------------------#
# Developer: The folloowing script block creats and add content to SharePoint 2013 Sites
#---------------------------------------------------------------------------------------#
#---------------------------------------------------------------------#
#1. Get all available Web Templates
#---------------------------------------------------------------------#
Get-SPWebTemplate

#---------------------------------------------------------------------#
#2. Get availablet Web Templates which are team site or black site
#---------------------------------------------------------------------#
Get-SPWebTemplate | Where {$_.Title -match "Team Site" -or $_.Title -match "Blank Site"}

#---------------------------------------------------------------------#
#3. Demo: Insert Command from Show-Command window
#---------------------------------------------------------------------#
Show-Command
#New-SPWeb -Url http://conteso.com:1234/capareademo1/Caparea_1 -AddToTopNav -Name Caparea_1 -Template STS#1 -UseParentTopNav
#New-SPWeb -Url http://conteso.com:1234/capareademo1/Caparea_2 -AddToTopNav -Name Caparea_2 -Template STS#1 -UseParentTopNav

#----------------------**** Important *****---------------------------#
#4. The Start-SPAssignment cmdlet properly disposes of objects used with variable assignments
# 'Global' Parameter- All objects are assigned to the lifetime of the script
# Read more at : https://technet.microsoft.com/en-us/library/ff607664.aspx
#---------------------------------------------------------------------#
Start-SPAssignment -Global

#---------------------------------------------------------------------#
#5. Create two SharePoint List of Type Tasks and Document Library.
#---------------------------------------------------------------------#
$web = Get-SPWeb -Identity http://conteso:1234/capareademo1/Caparea_1
$web.Lists.Add("Project Tasks", "Holds Project Tasks","Tasks")
$web.Lists.Add("Project Documents", "Holds Project Documents","DocumentLibrary")

#---------------------------------------------------------------------#
#6. Updating SharePoint Content
#---------------------------------------------------------------------#
$lst = $web.Lists["Project Tasks"]
$lst.Title = "Project Tasks 1"
$lst.Update()
#forget update

#---------------------------------------------------------------------#
#7. Remove SharePoint Content
#---------------------------------------------------------------------#
Remove-SPWeb -Identity http://conteso.com:1234/capareademo1/Caparea_1 -Confirm
Remove-SPWeb -Identity http://conteso.com:1234/capareademo1/Caparea_2 -Confirm

#-------------------------**** Important *****------------------------#
#4. The Stop-SPAssignment cmdlet properly disposes of objects used with variable assignments
# 'Global' Parameter- Disposed of when the Stop-SPAssignment cmdlet is called
# Read more at : https://technet.microsoft.com/en-us/library/ff607664.aspx
#---------------------------------------------------------------------#
Stop-SPAssignment -Global

0

Managed Metadata Services – SharePoint 2010/2013: Part 2/4 – Creating your first Managed Metadata Service Application

This blog post Part 2/4 is all about what Managed Metadata is for SharePoint, why is it so important to maintain accurate and consisted metadata across portals and how SharePoint can help manage metadata.

Part 1: Introduction
Part 2: Creating your first Managed Metadata Service Application
Part 3: Creating Term set, Terms and Tagging content with managed terms.
Part 4: Leveraging Termset to create and update Site Navigation

Step 1: Create new Managed Metadata Service using ‘New’ under ‘Service Applications’ section with in the ribbon.

Add new Managed Metadata Service

Add new Managed Metadata Service

Step 2: Give appropriate name to your service application, database name and you may create or use an existing application pool.


1 2

 Step 3: Confirm if the Service Application is created. Please see highlighted below.

3

 

Step 4: Make sure Managed Metadata Web Service is started under Central Administration > System Settings > Manage services on server

You may have encountered the following error. Please see the resolution following the screen capture.

5

Make sure your ‘Managed Metadata Web Service’ is started.

4

 Step 5: Reset IIS. Highly recommended.

You are all set! Congratulate yourself, you created  your first Managed Metadata Service.

Things to observe:

1. You should see at a database created with the name you gave to the Database in step 2.

Managed Metadata Service Database

2. Also there Should be an entry in the IIS, click explore and you should see a folder with .svc file.

Explore Managed Metadata IIS Entry

Please continue to read other related blog posts for more understanding of Managed Metadata services with SharePoint.

 

0

SharePoint Fast Search Concepts and Terminology – Part 4/4

This blog post is Part 4/4 of blog post series that will help you to get familiar with few concepts and terminologies referred in any search technology. As this blog series is more focused towards ‘Fast Search for SharePoint’ you may see jargon relevant to this.

Please check other related posts  Part 1| Part 2 | Part 3 | Part 4

The following graphic gives ten thousand foot view of what I am trying to capture and explain in this blog post series.

 

Fast Search Terminology and Concepts

8. Linguistics

Search engines perform a lot of language specific processing. This includes applying rules specific to that language and offensive language filtering, synonyms etc. Explaining in detail about how FS4SP behaves for languages is out of scope of this blog post.

9. Tokenization

When content sources are defined, Search engines crawl and will index those and will generate indexes. Some search engines refers to them as documents. Tokenization is a process of analyzing those documents and breaking the text into terms or tokens recognizable by the search engine.

10. Keyword Rank

This is the technique to improve ranking on documents that contains predefined keywords. This is a useful technique to forcefully enhance ranking among documents for better enhanced search experience.

11. Rank Profiles

Rank profiles are core component with FS4SP that defines how to rank items with in result set. There is a provision to create multiple Rank Profiles via Powershell, that defines how weights are applied to items or any components.

 

 

 

 

9. Keyword Rank

10. Rank Profiles

0

SharePoint Fast Search Concepts and Terminology – Part 3/4

This blog post is Part 3/3 of blog post series that will help you to get familiar with few concepts and terminologies referred in any search technology. As this blog series is more focused towards ‘Fast Search for SharePoint’ you may see jargon relevant to this.

Please check other related posts  Part 1| Part 2 | Part 3 | Part 4

The following graphic gives ten thousand foot view of what I am trying to capture and explain in this blog post series.

Fast Search Terminology and Concepts

7.Rank Tuning

By definition ‘Rank’ is the position in the hierarchy, in the search world ‘Rank’ is the position of the search result item in the result set. For instance, lets consider Bing or Google Search, when you search for ‘shoes’ or ‘vacation’ or in fact any search term you will find lot of search results. The first few results are the one user would click and finally end up being the revenue generators. So it is important to have your website show up in the first few results and is influenced by Rank. Seriously, how many times did you pass the first page of Google search results? So there are various technique that can be used to tune your Ranking. In the Fast Search world we have the following techniques.

7.1 Static/Quality Rank Tuning: URL Depth, Doc Rank, Site Rank, Hard Wired Boost

As the name indicates this the technique works independent of the search term. Which means that you would like to rank items independent of what user searches. Confused? OK, let me go through few examples and you would realize that you can tune Ranking independent of search terms.

URL Depth Ranking:
Observe  the below URLs and think about which of the two link, you would prefer to click?

1. http://www.sdakoju.wordpress.com/post/2015/mostpopular/fastsearchconfiguration

                                       OR

2. http://www.sdakoju.wordpress.com/FS4SPConfiguration

I would choose the second one, because it is more readable, clean and a more friendly URL. The depth of the URL is very less compared to the first one. ‘Depth’ indicates the importance or popularity of the page. The more deep the the page lies with in the site, the less popular the page is.  Obviously, you would always highlight your popular pages as your landing or one level below items. Search engines would like to show popular pages and not disliked or unknown pages lying some where deep down in the site.

Doc Rank:
If you have a page that is referenced across multiple pages, it influences the rank. Pages like these are ranked higher.

Site Rank:
This is similar to Doc Rank, the more links pointing the site or items with in the site the site has a higher rank

Hard Wired Boost:
Items can be give static ranking via Powershell and forcefully shown are high rank items

7.2 Dynamic Ranking:

This ranking value is based on query and its relation to the result set. This ranking is based on an multiple techniques/algorithms that influence ranking. Covering those is out of scope of this blog post. Please refer to Tune Dynamic Rank

 

0

SharePoint Fast Search Concepts and Terminology – Part 2/4

This blog post is Part 2/4 of blog post series that will help you to get familiar with few concepts and terminologies referred in any search technology. As this blog series is more focused towards ‘Fast Search for SharePoint’ you may see jargon relevant to this.

Please check other related posts  Part 1| Part 2 | Part 3 | Part 4

The following graphic gives ten thousand foot view of what I am trying to capture and explain in this blog post series.

Fast Search Terminology and Concepts

5. Recall and Precision

The total number of results in the result set for a query. You have to find a fair balance between Recall and Precision. The results set should not be too large and should avoid noise as much as possible. If the Recall is too large or too small it will hamper Precision.

There are various techniques to improve Recall such as Synonyms, Stemming etc.

Synonyms:

This is pretty common technique and a very obvious one. For instance, if you search for “happy” the search would also query for “joy”, “elated”, “merry” etc.

Stemming:

The use of Stemming is to get to the root form of a word. Stemming compares the root forms of the search terms to the documents in its content sources. For example, if the user enters “viewer” as the query, the search engine searches for “view” and returns all documents with view, viewer, viewing, preview, review etc

Recall and Precision Balance

Recall and Precision Balance

6.  Corpus

Corpus is Latin term for body. In the Search world, it refers to the scope of all the content sources the crawler would crawl and indexes. Following gives an example of what Corpus can include.

Corpus in Search

Corpus in Search

 

0

SharePoint Fast Search Concepts and Terminology – Part 1/4

This blog post is Part 1/4 of blog post series that will help you to get familiar with few concepts and terminologies referred in any search technology. As this blog series is more focused towards ‘Fast Search for SharePoint’ you may see jargon relevant to this.

Please check other related posts  Part 1| Part 2 | Part 3 | Part 4

Before we deep dive into these concepts, lets try to capture the overall process flow for any search engine. Typically content for any organization is stored in databases or documents on a file system. These documents can be of any type word, excel, images, videos, pdfs power-points etc. The primary job of any search technology is to crawl, index and surface results.

  1. Crawl:

    1. Once you have identified what content you want to crawl you will make search engine aware of where these are located and will grant appropriate permissions to crawl.  A crawl is basically collecting data and primarily metadata about the content.
  2. Index:

    1. Indexing is similar to how indexes work with books, they are just pointers to the actual location of the content. Typically indexes are physical files that are added to the file system and are output of a crawl.
  3. Search:

    1. Once the crawling and Indexing are complete its time to search these indexes, since these indexes are present on the file system querying these is lot faster.

The following graphic gives ten thousand foot view of what I am trying to capture and explain in this blog post series.

Fast Search Terminology and Concepts

Fast Search Terminology and Concepts

Lets get started:

1. Content Processing:

This is the nexus of any search engine, this defines what data-sources the search engine should crawl and the quality of the content itself for crawling. This is all about enriching the content even before it is being crawled.  Some of the tasks include

  1. Making sure that the search engine doesn’t crawl lot of noise consequently impacting the recall
  2. Detecting the language and applying rules
  3. Extracting meta data etc. and the list goes on.

2. Query Processing:

This kicks in after user performs the search, the search engine analyzes what the user is actually requesting and will accept additional query parameters if needed. This also matches result items in the search index  and returning search results to the user.

3. Relevancy:

This is the measure of how accurate/precise the search results are. There are various factors that determine how good the relevancy is, for instance the more the user find the intended results in the top search the better the relevancy is.  There are various techniques that can improve relevancy which will be discussed more in other part of this blog post.

4. Query Expansion: Best Bets, Synonyms, Lemmatization

Query expansion is a technique to improve Recall. The user may search for a term, search engines would not only search for specific term but also other relevant terms. This section explains on various techniques on how Query expansion can be accomplished.

4.1 Lemmatization

Lemma is a greek word which mean assumption or the canonical form of the word. For instance if the user searches for ship it would search shipped, ships etc. Not to be confused with Stemming where only the end of the word changes where it substitutes only the ending. For instance, Stemming would search for See, Seen, Seeing but no Saw. Where as Lemmatization would search for ‘Saw’ as well.

4.2 Synonyms

This is one of the most popular Recall technique, where Search engine would return results not only to the search terms but also for its synonyms. For instance if the user searches for ‘Joy’, it may include results for ‘Merry’, ‘Happy’, ‘Elated’, ‘Celebration’ etc.

4.3 Best Bets/Visual Best Bets

Best Bets are usually links displayed on the top of the search results pointing to different pages or content. These links are manually curated by administrator to display for a particular search term. Visual Best Bet similar to Best Bet except that an additional image is provided along with link and description.