Part 1: Setting up an S3 Reverse Proxy
Hybrid cloud environments consisting of on-premise, private cloud, and public cloud, have become the new norm for IT organizations as they simplify workload consolidation, scalability, recoverability and automation.
For IBM users, the IBM Cloud Direct Link (CDL) is a key technology for building hybrid clouds as it securely bridges an organization’s private network in IBM Cloud to their on-premises network so that resources and applications can work together as one.
One challenge for IBM users using IBM CDL is that the IBM Cloud Object Storage (COS) private cloud network is not directly accessible from systems on the on-premises side. To work around this issue an S3 reverse-proxy server must be configured within a given organization’s IBM Cloud infrastructure to enable those on-premises servers to access IBM COS without having to go over the public network.
Setting up a reverse proxy that is also highly available can be time-consuming to set up, monitor and maintain. This problem has been addressed with a new S3 Reverse Proxy management feature in QuantaStor 5.6 designed specifically for IBM COS by enabling IBM customers to set up a highly-available S3 reverse proxy in just a few clicks which we’ll go over in this article.
This enables hybrid cloud users to access their COS object storage data from IBM Cloud, IBM private cloud datacenters, as well as on-premise data seamlessly.
Further, QuantaStor 5.6 also provides users with the ability to present buckets within IBM COS as NAS storage which can be accessed via traditional protocols including NFS and SMB.
Basic S3 Reverse Proxy Setup with QuantaStor 5.6
In this article we’ll first go over the simple case of setting up a single QuantaStor server as an S3 Reverse Proxy for IBM Cloud Object Storage. Then in the following sections, we’ll go into adding a second server and making the proxy highly available.
Step 1) Provision one QuantaStor server for use as your S3 Reverse Proxy server.
Step 2) Navigate to the Cloud Integration tab, then select the S3 Proxy section.
Step 3) Create the S3 (Reverse) Proxy by clicking the “Create” button.
Step 4) Configure your DNS server or /etc/hosts files to resolve the server name you applied to the proxy to resolve to the QuantaStor server IP address.
Step 5) Test the proxy by using a web browser on a server or workstation within your IBM CDL network to point to the server FQDN you assigned to the proxy.
Provisioning a QuantaStor server in the IBM Cloud
The QuantaStor S3 reverse proxy setup requires at a minimum one QuantaStor server (two to enable high-availability). Hardware requirements for QuantaStor servers used as an S3 Proxy are minimal with just one Intel Xeon processor (8 core minimum), 32GB or more of RAM, and two 200GB (or larger) SSDs for use as the RAID1 mirrored boot drive for the QuantaStor OS.
- 1x or 2x servers with Quantastor 5
- (2x required for highly-available proxy, 1x for basic non-HA proxy)
- 32GB or more RAM per server
- 8 processor cores or more per server
- Dual 10GbE ports on Private Network recommended but dual 1GbE will be sufficient if your IBM CDL link to the IBM Cloud is 1Gb or less.
- 2x SSDs for RAID1 mirrored boot (200GB or larger)
You may deploy QuantaStor systems in any IBM Cloud datacenter world-wide and QuantaStor appears as an operating system choice on most bare metal server options when you go through the IBM Cloud provisioning portal. Simply choose QuantaStor 5 with the XS (extra-small) license as there is no metering of the capacity used via QuantaStor’s S3 reverse proxy.
Creating an S3 Reverse Proxy
Once your QuantaStor server is deployed, login to it using your web browser by going to the IP address assigned to the QuantaStor system. Login as “admin” with the generated password provided via your IBM Cloud account’s system management portal.
Once logged into the system, navigate to the top tab marked “Cloud Integration” then choose the S3 Proxies section on the left. This will update the toolbar with some new options, choose “Create” to make a new proxy.
In the screenshot above you can see that we’ve named our proxy “proxy1” and given it a server name of “s3-api.dal-us.geo.example.com”. Only when the QuantaStor system is addressed via this virtual host name will it act as a proxy for S3 calls to the selected IBM COS endpoint. In this example we’ve selected to have the hostname “s3-api.dal-us.geo.example.com” route to “s3-api.dal-us.geo.objectstorage.service.networklayer.com”. The “networklayer.com” portion of the endpoint tells us that this is a private network endpoint to the IBM COS storage. Do not choose the “softlayer.net” endpoints as those route over the public network.
Configure your DNS server
In the sample above we assigned our new S3 Proxy to have a fully-qualified-domain-name (FQDN) of “s3-api.dal-us.geo.example.com” but you’ll want to give it a FQDN that is resolved by your internal DNS servers to route to the IP address of the QuantaStor server. It can be a detailed FQDN like the example used here or could be something simple like “s3api.mycompany.com”. An easy way to test that the proxy is working if you don’t have access to edit your DNS server configuration is by editing the /etc/hosts file (c:\Windows\System32\Drivers\etc\hosts on Windows 10) to add an entry for the FQDN that has the IP address of the QuantaStor server.
Testing the S3 Reverse Proxy
An easy check to verify that the proxy is working is to just input the FQDN you’ve assigned to the proxy into your favorite web browser. To continue the example above where we chose the proxy server FQDN to be “s3-api.dal-us.geo.example.com” you can enter that or better use the URL “https://s3-api.dal-us.geo.example.com”. This will route to your QuantaStor server and it will see the “s3-api.dal-us.geo.example.com” FQDN used in the https request and will in turn route that request to/from IBM COS as your proxy to the selected endpoint. You should see a warning page to accept the generated self-signed keys and then once through that you’ll see a message from the IBM COS storage like this indicating “GET Service not allowed for anonymous users”.
Have Questions or Need Assistance?
IBM Cloud and OSNEXUS support staff are available to help you configure and verify your setup. To get assistance, open an IBM support ticket through the portal to request assistance with escalation to OSNEXUS and a support engineer will assist.
In our next blog post, we will go over how to set up a highly-available S3 Reverse Proxy cluster.
Categories: Storage Appliance Hardware
www | grep storage 
Leave a comment