RemoteBlobStorage Providers in machine.config

Sep 22, 2008 at 10:02 AM
Edited Sep 22, 2008 at 1:22 PM

Hi,

Please consider a scenario where a table has 2 blob columns blob1 and blob2. I have to store blob1 using a web service and blob2 on a WebDAV server. How can we achieve this?

I could see the RemoteBlobStorage setting in machine.config with Providers section. Can you list out the other set of providers available, with their configurations like the given one? 
 <Providers>
   <file>
    <add key="CreateNewBlobMethodImplemented" value="1" />
    <add key="EnumerationOptimizationLevelImplemented" value="1" />
    <add key="BlobStoreType" value="File" />
    <add key="DllFile" value="D:\SQL Server 2008\Source\CS\InstallProvider\bin\Debug\Microsoft.Data.BlobStores.FileBlobStore.dll" />
    <add key="ClassName" value="Microsoft.Data.BlobStores.FileBlobStore.FileStoreLibrary" />
    <add key="ProviderVersion" value="1.0.0.0" />
    <add key="MinSupportedBackEndStoreVersion" value="1.0.0.0" />
    <add key="BlobStoreAbstractClassVersion" value="10.0.0.0" />
   </file>
 </Providers>

 

 

How does the SQL Server 2008 use the entries that are made by using the SP "mssqlrbs.rbs_sp_add_blob_store" in table "mssqlrbs_resources.rbs_internal_blob_stores", to notify the registered blob stores? Here blog_store_location seems to be a UNC path. How do we handle situations like URI or a custom end point (web service, WebDAV) to be used as file store.

Thanks in advance
Saju Mathew

 

 

 


 

Editor
Sep 22, 2008 at 8:08 PM

Without writing a dedicated RBS provider (or modifying the FileStore provider sample) for your web service or WebDAV server I don't think you'll be able to make this work. The File Store provider is specifically for using UNC paths or local directories.

The machine.config entries just register the provider dll so the RBS client can load the assembly. All configuration is done via the add blob store stored proc - you can have a single machine.config entry and multiple blob store instances of the same type with different configuration options passed in through the add blob store sp. If you have 2 different blob stores you wish to connect to and the appropriate provider for each, just register two blob store entries using the add blob store procedure.

Unless you are able to connect to these stores using UNC paths and impersonation, RBS with the sample File Store provider won't work. This is however why the sample code has been posted, so you can customize for specific blob stores. Please let me know if I can help further or you need more detail.

- mike

Sep 23, 2008 at 5:06 AM
Mike,

We are very clear on your first point; extending the file store provider to support the custom stores. 

But in the second point, are you talking about a factory pattern handling various data stores, by using the custom settings, provided in "extended_configuration" column of  "mssqlrbs_resources.rbs_internal_blob_stores" table?

In brief how can the RBS API support custom providers from multiple vendors on single box?

Thanks,
-Saju
Editor
Sep 23, 2008 at 5:06 PM
There's a provider library spec up on MSDN that might give some more detail.

http://msdn.microsoft.com/en-us/library/cc905212.aspx

To answer your question, there's two main concepts here - the provider dll itself and the blob store instance. There can be multiple provider dlls registered with RBS - this will mean there's multiple entries in the machine.config file. This is what the FileProviderConfiguration.xml file is. If you create two new provider libraries you'll need to create this xml file with the standard config items filled out (ConfigurationItem tags) plus any provider specific config keys you may need for each one - they're just defined in the xml file, the actual values for the instance of the blob store are passed in during the sp add blob store registration. The sample File provider has one config key, pool_capacity of type Number (other options are Name, Boolean, Binary and Duration) but you can define as many as you need here. The RegisterFileBlobStore.sql file gives an example of actually adding the blob store to the RBS instance. This contains an xml with the configuration info that's passed to rbs_sp_add_blob_store which actually adds the blob store as a new store visible in the mssqlrbs.rbs_blob_stores view. You're right this is a factory pattern - during RBS startup we reflect through the list of dlls registered in the machine.config file, then query the rbs_blob_stores view and create one blob store instance per row in rbs_blob_stores. The type of instance is specified in rbs_blob_stores and this is the class we actually instantiate.

So, to have custom providers from multiple vendors, you need to register each provider dll in the machine.config file, then add one or more blob store instances of each type to the RBS configuration tables using the rbs_sp_add_blob_store method. Then your app can pick which provider to store by passing in a ConfigItemList with the BlobStoreName parameter set.

As part of these codeplex samples there's a generic utility to install a provider. Take a look at InstallProvider.cmd for details on calling this.

Hope this helps, let me know if you need more details.

- mike
Editor
Sep 24, 2008 at 4:25 PM
Hi Saju,

Can you please tell us more about what you are trying to achieve?

Windows clients have WebDAV redirector, so a WebDAV share should be accessible using a UNC path. So, you should be able to use the sample RBS File Store provider to store blobs in WebDAV.

Thanks,
Pradeep.
Sep 25, 2008 at 2:07 PM
Edited Sep 25, 2008 at 2:10 PM
Hi Pradeep,

What I am trying to achieve is to develop custom RBS Providers for storing blob data on different data storage systems.
The RBS Provider interacts with data storage systems by means of:
  • Web Services
  • WCF Service
  • SMB (CISF) mounted volumes
  • WebDAV share
  • REST 

Every provider might have its own set of configurations like end point(s), that may be stored in "mssqlrbs_resources.rbs_internal_blob_stores" using the SP "rbs_sp_add_blob_store". The existing File Store Provider may be extended to support CISF and WebDAV(provided WebClient service is up and running for WebDAV). For the other cases listed above, we have to introduce custom RBS providers.
machine.config doesn't accept more than one <file> config; when tried, throws error "Sections must only appear once per config file"; because it is tied to the schema using 
<sectionGroup name="RemoteBlobStorage" type="System.Configuration.ConfigurationSectionGroup, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" >

Is there a way where we can register multiple RBS providers in machine.config? If possible how do RBS API know, which provider (DllFile and ClassName) need to be invoked/notified?

If not possible how can we support RBS Providers from multiple vendors? This may be required for storing blob data of different tables, databases or sql server instances on remote data storage systems provided by different vendors.

Please consider a case where a table has 2 blob columns blob1 and blob2. blob1 data may be stored using a web service and blob2 on a server with REST protocol support. In this case how can we map the columns to the respective RBS provider?

Please let me know if you need more information.

Thanks,
-Saju

Editor
Sep 25, 2008 at 5:44 PM
The misunderstanding here is the relationship between the provider dll information stored in the machine.config and the blob store configuration added using rbs_sp_add_blob_store.

Each dll (ie each custom version of the provider you create) only needs to be registered once in machine.config. You can then add as many blob stores of that type to the rbs instance using the add blob store proc.

If you create a custom web service dll the name won't be file, it will be your custom name. So you can add multiple provider dlls to the machine config file, then have multiple blob store instances of each type in a single RBS instance.

- mike
Sep 29, 2008 at 11:02 AM
Mike,

I was unable to add the other custom providers like <file> in machine.config due to the missing entry in <sectionGroup>

<sectionGroup name="Providers" type="System.Configuration.ConfigurationSectionGroup, System.Configuration, 
            Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" >
    <section name="file" type="System.Configuration.AppSettingsSection, System.Configuration, 
            Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</sectionGroup>

The configuration related issue has been resolved and the points you have mentioned through out the discussion are clear to me.

Thanks a lot for you time and inputs. Hope I can trouble you if I am held up :)

Thanks,
-Saju