CodePlex Provider and SharePoint 2010 (RTM)

Jul 7, 2010 at 11:13 PM
Hi Guys... I've got a tough one that I've been fighting for a month or two now. I originally installed the RBS Provider from this site (after a recompile) to SharePoint 2010 (beta) and had very few problems getting it up and running. I first noticed that it wasn't working with the RTM version a month or so ago but didn't have much time to look into it. So for the last 3 days I've been applying every troubleshooting technique I can think of (ULS Logs, Developer Dashboard, exposing page level errors, Event log, and more!) to try to figure out what's going on. I realize that this is technically a SQL Server based blog, but I'm hoping you guys have worked with the SharePoint team enough to help. Here's the deal. After installing the provider, I'm able to verify that the configuration is correct by running the "[SPRemoteBlobStorageSettings].Migrate()" method in the SharePoint 2010 Management shell (PowerShell). I'm able to send the existing binaries in the content database to the blob store no problem. The binaries show up and I can verify that I get an RBSID in the AllDocStreams table. I can also use PowerShell to set the active provider to "" [emptystring] and run Migrate() again and get all content to internalize back to the content database. So the provider is functional and appears to be configured correctly. So now the problem. With the provider enabled, I go to the SharePoint UI via the browser and try to upload a file. Right when I execute the upload, I get a nasty error message that says "The URL 'Documents/3P0.tif' is invalid. It may refer to a nonexistent file or folder or refer to a valid file or folder that is not in the current web." It took a bit of sleuthing but I eventually identified the following stack trace: [SPException: The URL 'Documents/3PO.tif' is invalid. It may refer to a nonexistent file or folder, or refer to a valid file or folder that is not in the current Web.] Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx) +27257906 Microsoft.SharePoint.Library.SPRequest.PutFile(String bstrUrl, String bstrWebRelativeUrl, Object punkFile, Int32 cbFile, Object punkFFM, PutFileOpt PutFileOpt, String bstrCreatedBy, String bstrModifiedBy, Int32 iCreatedByID, Int32 iModifiedByID, Object varTimeCreated, Object varTimeLastModified, Object varProperties, String bstrCheckinComment, Byte partitionToCheck, Int64 fragmentIdToCheck, String bstrCsvPartitionsToDelete, String bstrLockIdMatch, String bstEtagToMatch, Int32 lockType, String lockId, Int32 minutes, Int32 fRefreshLock, Int32 bValidateReqFields, Guid gNewDocId, UInt32& pdwVirusCheckStatus, String& pVirusCheckMessage, String& pEtagReturn, Byte& piLevel, Int32& pbIgnoredReqProps) +27637951 Microsoft.SharePoint.SPFileCollection.AddStreamOrBytesInternal(String urlOfFile, Stream file, PutFileOpt fileOpt, String createdBy, String modifiedBy, Int32 createdByID, Int32 modifiedByID, DateTime timeCreated, DateTime timeLastModified, Object varProperties, String checkInComment, Stream formatMetadata, String lockIdMatch, String etagToMatch, SPLockType lockType, String lockId, TimeSpan lockTimeout, Boolean validateRequiredFields, SPVirusCheckStatus& virusCheckStatus, String& virusCheckMessage, String& etagNew, Boolean& ignoredRequiredProps) +26283940 Microsoft.SharePoint.SPFileCollection.Add(String urlOfFile, Stream file, SPFileCollectionAddParameters parameters) +432 Microsoft.Office.RecordsManagement.PolicyFeatures.ApplicationPages.UploadPage.UploadFile(String& leafName, SPVirusCheckStatus& checkStatus, String& virusMessage) +812 Microsoft.Office.RecordsManagement.PolicyFeatures.ApplicationPages.UploadPage.UploadFile() +78 Microsoft.Office.RecordsManagement.PolicyFeatures.ApplicationPages.UploadPage.OnSubmit(Object o, EventArgs e) +332 Microsoft.Office.RecordsManagement.PolicyFeatures.ApplicationPages.UploadExPage.OnSubmit(Object o, EventArgs e) +255 System.Web.UI.HtmlControls.HtmlInputButton.OnServerClick(EventArgs e) +115 System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument) +29 System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +2981 So my problem is that I can only trace this bad boy as far as the "PutFile" COM call. At that point, I lose my trail into the hidden abyss known as COM! [I'm not smart enough to debug unmanaged code! ;-) ] But I can run the RBS Provider project and attach to various SharePoint processes (w3wp.exe and even owstimer.exe for good measure) and all indications are that the RBS Provider DLL is never even invoked. So I can't tell if I've got some sort of permissions problem or if there is something else wierd going on inside of COM. But something seems to be preventing my provider from being invoked such that the document is allowed to be externalized and the metadata sent on to SharePoint. So then the response seems to be "it doesn't exist". Do you have any advice? Or can you perhaps talk to someone on the "SharePoint RBS" implementation team that might have an idea as to what might be going south? I have gained a pretty solid understanding of RBS in the last 6 - 9 months. I'm going to be presenting on the topic at a SharePoint Saturday in a few weeks. I would REALLY like to demo to the attendees the lifecycle of a file upload/retrieval with RBS enabled. But I can't do that if SharePoint won't cooperate. I can fallback onto the FILESTREAM provider if I have to. But having the source code to a working provider is tremendously powerful for helping to understand what is going on inside of the various features of SharePoint 2010. Just trying to get the word out on this stuff! ;-) Best Regards, Russ
Jul 7, 2010 at 11:14 PM
Holy lost formatting batman!
Editor
Jul 8, 2010 at 6:53 PM

Ok, so just to reiterate my understanding of what's happening:

  • Loading and storing files via SharePoint into the RBS Filestream provider is working well.
  • You've rebuilt the demo File Share provider and can get it working with the rebuilt demo application fine.
  • After installing and enabling the File Share provider in SharePoint any put blob operation is failing.

If the above is the case there's a couple of things I can think of that might be going wrong. Can you check the following:

  • The File Share provider is being built against .NET 2.0 or .NET 3.5SP1 rather than .NET 4.0
  • The File Share provider has been successfully added to the GAC
  • Can you turn on the Fusion log to see if anything interesting pops up, instructions here:

Just a couple of quick thoughts that might help. Good luck!

- Michael

 

 

Jul 8, 2010 at 9:01 PM
Edited Jul 8, 2010 at 9:02 PM

Michael, So here is where I'm at.  First, to clear a couple things up:

- When enabling RBS in SharePoint we use PowerShell in the final step. After acquiring a reference to the ContentDatabase.RemoteBlobStorageSettings object with an $rbss variable, I'm able to execute $rbss.Migrate(). This invokes the File ShareProvider assembly in the context of PowerShell and successfully externalizes the binaries to the BLOB Store, leaving only the RBSID in SharePoint.

- So with RBS, then enabled on my site collection (content database), I use the browser to navigate to the site collection. At this point, if I try to retrieve a document that is now externalized in the Blob Store, SharePoint can't find the file and returns a 404. Further, if I try to upload a new file, I get the nasty error I provided above.

So RBS appears to be properly configured or else the PowerShell commands would fail. However, SharePoint is unable to use the File Share provider assembly when attempts are made through the browser and ultimately IIS. This would indicate to me some sort of permissions or CAS policy issue. So just to try to hit the fly with a hammer, I added my assembly to the GAC, set SharePoint to run with Full trust (always a no-no), and even granted "Everyone" access to the BLOBStore share. I would do none of these things in the real-world but I'm just trying to figure this out.

To answer your other questions, the File Share provider is currently compiled against .NET 3.5SP1. Also, I took a crack at the Fusion log view with the appropriate registry keys to show broken bindings and got nothing from that either. Any other thoughts? I'm about to give up and just use FILESTREAM for the demo. That would be unfortunate because I have all kinds of other things I want to test using this provider. Thanks for taking the time! - Russ

Editor
Jul 8, 2010 at 9:11 PM

Can you try adding NT AUTHORITY\NETWORK SERVICE (usually this is the ASP.NET process account) to the file share? It's not actually a member of Everyone. If this still fails there might be some breadcrumbs in the event log, either application or security to see what attempts are being made.

Editor
Jul 15, 2010 at 4:23 PM

Did you get any further with this Russ? Once we've determined if it's an access issue we can work on using RBS's secure credential management features to lock down the share correctly.

Jul 26, 2010 at 10:42 AM

Hi Guys,

I'm trying also to get the File provider installed and got the same problem as Russ.

Event Viewer, RBS message:

Message ID:4, Level:ERR , Process:4452, Thread:7 An error occurred while reading extension file C:\Program Files\RBSFile\Microsoft.Data.BlobStores.FileBlobStore.dll. System.IO.FileLoadException: Could not load file or assembly 'Microsoft.Data.BlobStores.FileBlobStore, Version=10.0.0.0, Culture=neutral, PublicKeyToken=f429ee06e80a9fba' or one of its dependencies. Failed to grant permission to execute. (Exception from HRESULT: 0x80131418) File name: 'Microsoft.Data.BlobStores.FileBlobStore, Version=10.0.0.0, Culture=neutral, PublicKeyToken=f429ee06e80a9fba' ---> System.Security.Policy.PolicyException: Execution permission cannot be acquired. at System.Security.SecurityManager.ResolvePolicy(Evidence evidence, PermissionSet reqdPset, PermissionSet optPset, PermissionSet denyPset, PermissionSet& denied, Boolean checkExecutionPermission) at System.Security.SecurityManager.ResolvePolicy(Evidence evidence, PermissionSet reqdPset, PermissionSet optPset, PermissionSet denyPset, PermissionSet& denied, Int32& securitySpecialFlags, Boolean checkExecutionPermission) at System.Reflection.Assembly.nLoadFile(String path, Evidence evidence) at System.Reflection.Assembly.LoadFile(String path) at Microsoft.Data.SqlRemoteBlobs.ProviderClass.Initialize() Operation: AssemblyLoad BlobStoreId: 0 Log Time: 7/26/2010 9:45:57 AM

 

Output from Fuslogvw.exe

*** Assembly Binder Log Entry (7/26/2010 @ 11:45:57 AM) ***

The operation was successful.

Bind result: hr = 0x0. The operation completed successfully.

Assembly manager loaded from: C:\Windows\Microsoft.NET\Framework64\v2.0.50727\mscorwks.dll

Running under executable c:\windows\system32\inetsrv\w3wp.exe

--- A detailed error log follows.

LOG: IJW explicit bind. File path:C:\Program Files\RBSFile\Microsoft.Data.BlobStores.FileBlobStore.dll.

LOG: IJW assembly bind returned file not found.

 

Well the provider is in that directory C:\Program Files\RBSFile !! And Everyone has access to the directory also the Network Service account, same rights on the \\server\share

What is goes wrong here?

Best Regards,

J

 

 

Editor
Jul 26, 2010 at 4:59 PM

I think this might be an issue related to signing of the provider library rather than the same issue that Russ is hitting, I believe his provider is loading correctly however it can't access the share given. Looks like your provider dll is never being loaded, so this indicates an earlier problem.

Have you signed your provider (either strong named signed or used sn.exe to provide a strong name signing override) and registered the DLL in the GAC?

Jul 27, 2010 at 2:08 AM

My apologies for the delayed response.  I've been on vacation.  While I was away, I received the following information in a direct contact email:

"I figured it out.  The account that is running the application pool for the web site needs to be given the SQL Server Role for the Blob database called "db_rbs_admin".  After I did that, had no problems."

I'll be testing the fix myself in the next couple of days and I'll reply with my results.  Hopefully we've found the solution.

Editor
Jul 27, 2010 at 4:16 PM

Great Russ. I've added a note to the front page of the codeplex site detailing this.

Aug 2, 2010 at 2:45 PM

Update...

Heading down the permissions track, I've been able to get the provider to function properly now in SP2010 RTM.  The problem is, that I still can't isolate the exact issue.  My demo VM is a fully contained Domain Controller, SQL Server, and SP2010 server.  I'm able to resolve the issue by making the SP web application app pool account a domain administrator.  Clearly the app pool account is very important as it is the context that is used to store the document in the BLOB store.

I realize that this is hitting the fly with a hammer and is clearly not ideal in a "least priviledge" environment.  I think some of my complication is that I have so much going on in a single VM.  I'm going to try a standalone implementation (no domain) and see if I can further isolate the issue.  Eventually, I'd like to try a multi-server environment with a Domain Controller, but I need to clear some VMs from my VM host before I can do that.

The good news is that it works.  But I do still need to further investigate the exact permission that is causing the problem.

On a side note... Mike (since I'm pretty sure you're going to read this)... Do you know why we have to create a master encryption key in SQL Server (command below)?  How is that used by RBS?  I'm just trying to understand everything from beginning to end so that I can troubleshoot these types of issues.

IF NOT EXISTS (SELECT * FROM sys.symmetric_keys WHERE NAME = N'##MS_DatabaseMasterKey##') CREATE MASTER KEY ENCRYPTION BY PASSWORD = N'Admin Key Password !2#4'

Thanks!

Editor
Aug 5, 2010 at 10:18 PM

Sorry it took so long to get back to you Russ, I wanted to put together some detailed information on this for everyone. The most recent blog post on the msdn RBS site should give a fairly complete explanation.

http://blogs.msdn.com/b/sqlrbs/archive/2010/08/05/rbs-security-model.aspx

Let me know if you have any questions or need clarification on this.

cheers

mike

 

May 16, 2011 at 5:01 AM
Edited May 16, 2011 at 5:40 AM

i hope that this thread is still alive...

i am having document upload issue and i am getting the same error....

in windows application event log i have following entry

Message ID:3, Level:ERR , Process:7380, Thread:19
Exception thrown:
Operation: Exception
BlobStoreId: 0
Log Time: 5/16/2011 4:04:30 AM
Exception: Microsoft.Data.SqlRemoteBlobs.RemoteBlobConfigurationException: No provider of type <File> found. Check the server configuration or install the provider on the client.
ExceptionType:      Configuration
ExceptionCode:      ProviderInterfaceNotFound
Request Information not set.
Session Information not set.
Provider Session Information:
Provider Session Info:
ProviderSession:         ProviderSession. Id: <3>, Name: <CodePlex_Provider_1>, Type: <File>, StoreVersion: <1.0.0.0>, Location: <C:\RBSStore\>
Extended Configuration:

and Sharepoint site error

The URL 'Shared Documents/File.docx' is invalid.  It may refer to a nonexistent file or folder, or refer to a valid file or folder that is not in the current Web.

Troubleshoot issues with Microsoft SharePoint Foundation.

Correlation ID: c95c16e2-f9e9-44af-ba59-c3b4abd256fe

Date and Time: 5/16/2011 9:42:39 AM

i have single system Sharepoint Farm environment where SQL 2008 R2 server and Sharepoint 2010 Server are installed on the same system...

please guide me if there is any resolution for the same.

 

thanks and regards.

sarjan

 

May 17, 2011 at 3:49 PM

There a lot of potential causes to this error.  It is usually caused by a missed step during provider deployment.

Here are a few things to check:

  • The RBS provider is loaded into the GAC to avoid any code execution issues
  • The app pool of the web application servicing the content database in question... has full read/write access to the BLOB store.
  • The RBS provider was not properly "enabled" using PowerShell

Hope that helps!

 

 

May 18, 2011 at 10:55 AM

rhouberg,

first of all thanks a lot for real fast reply....

As you suggested I followed the checks...

  • The RBS provider is loaded into the GAC to avoid any code execution issues

         i did find the Assembly installed still for assurance I reinstalled it using gacutil .

  • The app pool of the web application servicing the content database in question... has full read/write access to the BLOB store

        IIS App pool has full access to content DB.

  • The RBS provider was not properly "enabled" using PowerShell

       here i must have done something must have been gone wrong. when i recompiled the provided code with my added functionality and installed the provider the error has disappeared.

thank you very much for guiding me towards required direction.....

i have a question...

Is it possible that this error appears when some wrong piece of code is added to provider code and this compiled DLL is loded to GAC for RBS to use?