Connectors
The platform GUI provides a utility called a Service Connector to sync data between the platform's cloud-hosted data store and a users local machine, which can be your computer or server. This small piece of software will allow you to securely upload data or download results using TLS 1.2. The connector makes 2 connections:
A control connection, which the Connector uses to get its configuration info from the platform, and to update the platform about its activities
A connection towards the storage node, used to upload sample data to Illumina, or download results files to you
This Connector runs in the background, and configuration is done in the Illumina Connected Analytics platform, where you can add upload and download rules to meet the requirements of the current project and any new projects you may create.
Create a New Connector
Select a project.
From the projects menu, select Connectivity.
Select Request New or + Add Connector.
Enter required connector fields.
Unique name — Enter the name of the connector.
Description — Enter the connector description.
Mode — Specify if the connector can upload data, download data, or neither.
Technical Code — Copy the technical code. The installer requires the code to connect to Illumina Connected Analytics interface.
Operating system — Select your server or computer operating system.
Add any upload or download rules. See Connector Rules.
Select Install and wait for the installer to download. An initialization key will be displayed in the platform now.
Launch the installer after download and follow the on-screen prompts to complete the installation. Be sure to copy the initialization key from your browser for use in the installation process before closing the tab or window.
Windows Run the downloaded .exe file. During the installation, the installer will ask for the initialization key. Fill out the initialization key you see in the platform. The installer will create an Illumina Service Connector, register it as a Windows service, and start the service. That means, if you wait for about 60 seconds, and then refresh the screen in the Platform by using the refresh button in the right top corner of the page, the connector should display as connected. You can only install 1 connector on Windows. If for some reason, you need to install a new one, first uninstall the old one. You only need to do this when there’s really a problem with your existing connector. Upgrading a connector is also possible. To do this, you don’t need to uninstall the old one first.
OS X Double click the downloaded .dmg file. Double click Illumina Service Connector in the window that opens to start the installer. Run through the installer, and fill out the initialization key when asked for. To start the connector, once installed or next time, you open the app. You can find the app on the location where you installed it. The connector icon will next appear in your dock when the app is running. In the platform on the Connectivity page you can check whether your local connector has been connected with the platform. This can take 60 seconds after you started your connector locally. You should refresh the Connectivity page to see the latest status of your connector. The connector app needs to be closed to shut down your computer. You can do this from within your dock.
Linux Installations require Java SE 8 or 11. You can check this with ‘java –version’. With Java installed, you can run the installer. Depending on whether you have an X server running or not, it will display a UI, or follow a command line installation procedure. You can force a command line installation by adding a –c flag. You can start the connector from within the BSC home directory with ‘./illuminaserviceconnector start’.
Connector Rules
In the upload and downlaod rules you define different properties when setting up a connector. Most importantly you can indicate to which project data needs to be uploaded or to which location data needs to be downloaded. A connector can be used by multiple projects and a connector can have multiple upload and download rules. Configuration can be changed anytime. Changes to the configuration will be applied approximately after 1 minute, when the connector is connected, to any new data transfers that will be scheduled. The following are the different properties you can configure when setting up a connector. After adding an upload rule and installing the connector, you can use the Active Flag to disable rules.
Upload Rules
An upload rule tells the connector which folder it needs to watch on your disk. The connector contacts the platform every minute to pick up changes to upload rules.
| Field | Description |
|---|---|
Unique name | Name of the upload rule. |
Local folder | The path to the folder where files to be uploaded are stored. |
File pattern | Specify the files to upload. |
Location | The location the data will be uploaded to. |
Project | The project the data will be uploaded to. |
Description | Additional information about the upload rule. |
Type of Files | the files need to get inside the platform. |
Data owner | The owner of the data after upload. |
Advanced Rules
Select Advanced to configure the following optional advanced upload rule settings.
| Field | Description |
|---|---|
Combine files into samples | Combine multiple files into one sample. |
Automatically start pipeline | Automatically start pipelines after completing the upload. Only possible for pipelines that require a single file as input. |
Download Rules
When you schedule downloads in the platform, you can choose which connector needs to download the files. That connector needs some way to know how and where to it needs to download your files. That’s what a download rule is for. The connector contacts the platform every minute to pick up changes to download rules. The following are the different download rule settings.
| Field | Description |
|---|---|
Order of execution | If using multiple download rules, set the order the rules are performed. |
Unique name | Name of the download rule. |
Target Local folder | The path to the folder where the files must be downloaded to. |
Description | Additional information about the download rule. |
Project | The projects the rule applies to. |
Format | The format the files must comply to in order to be scheduled as downloaded. |
Advanced Rules
Select Advanced to configure the following optional advanced upload rule settings.
| Field | Description |
|---|---|
Filename Expression | Edit the file names of your downloaded files for pipelines with auto-download enabled. |
Disable hashing | Disable validity checks when downloading data. Data download is faster but could potentially result in corrupt files. |
Shared Drives
When you set up your connector for the first time, and your sample files are located on a shared drive, it’s best to create a folder on your local disk, put one of the sample files in there, and do the connector setup with that folder. When this works, try to configure the shared drive.
Transfer to and from a shared drive may be quite slow. That means it can take up to 30 minutes after you configured a shared drive before uploads start. This is due to the integrity check the connector does for each file before it starts uploading. The Connector can upload from or download to a shared drive, but there are a few conditions:
• The drive needs to be mounted locally. X:\illuminaupload or /Volumes/shareddrive/illuminaupload works, \shareddrive\illuminaupload or smb://shareddrive/illuminaupload doesn’t • The user running the connector must have access to the shared drive without a password being requested. • The user who runs the Illumina Service Connector process on the Linux machine needs to have read, write and execute permissions on the installation folder.
Update connector to newer version
Illumina might release new versions of the Service Connector, with improvements and/or bug fixes. You can easily download a new version of the Connector with the Download button on the Connectivity screen in the platform. After you downloaded the new installer, run it and choose the option ‘Yes, update the existing installation’.
Uninstall a Connector
To uninstall the connector, perform one of the following:
Windows and Linux: Run the uninstaller located in the directory the connector was installed.
Mac: Move the Illumina Service Connector to your Trash folder.
Troubleshooting
Log files
The Connector has a log file containing technical information about what’s happening. When something doesn’t work, it often contains clues to why it doesn’t work. Interpreting this log file is not always so easy, but it can help the support team to give a fast answer on what’s wrong, so attaching it to your email when you have upload or download problems is often a good idea. You can find this log file at the following location:
| Operating system | Location |
|---|---|
Windows |
|
OS X |
|
Linux |
|
Common problems
| Operating system | Issue | Solution |
|---|---|---|
Windows | Service connector doesn't connect | First, try restarting your computer. If that doesn’t help, open the Services application (By clicking the Windows icon, and then typing services). In there, there should be a service called Illumina Service Connector. • If it doesn’t have status Running, try starting it (right mouse click -> start) • If it has status Running, and still doesn’t connect, you might have a corporate proxy. Proxy configuration is currently not supported for the connector. • If you don’t have a corporate proxy, and your connector still doesn’t connect, create a support ticket, and include your connector log files. |
OS X | Service connector doesn't connect | Check whether the Connector is running. If it is, there should be an Illumina icon in your Dock. • If it doesn’t, log out and log back in. An Illumina service connector icon should appear in your dock. • If it still doesn’t try starting the Connector manually from the Launchpad menu. • If it has status Running, and still doesn’t connect, you might have a corporate proxy. Proxy configuration is currently not supported for the connector. • If you don’t have a corporate proxy, and your connector still doesn’t connect, create a support ticket, and include your connector log files. |
Linux | Service connector doesn't connect | Check whether the connector is running ps aux | grep illumina • If it doesn’t, try starting the service sudo service illuminaserviceconnector start • If it has status Running, and still doesn’t connect, you might have a corporate proxy. Proxy configuration is currently not supported for the connector. • If you don’t have a corporate proxy, and your connector still doesn’t connect, create a support ticket, and include your connector log files. |
Linux | Can’t define java version for connector | The connector makes use of java version 8 or 11. If you run the installer and get the following error “Please define INSTALL4J_JAVA_HOME to point to a suitable JVM.”: • When downloading the correct java version form Oracle, there are there are 2 variables in the script that can be defined (INSTALL4J_JAVA_HOME_OVERRIDE & INSTALL4J_JAVA_PREFIX) , but not INSTALL4J_JAVA_HOME, which is printed in the above error message. Instead, export the variable to your env before running the script. If you get the following error message “gzip: sfx_archive.tar.gz: not in gzip format. I am sorry, but the installer file seems to be corrupted. If you downloaded that file please try it again. If you transfer that file with ftp please make sure that you are using binary mode.” : • Editing the shell file makes it corrupt. Instead, export the variable to your env before running the script. You can export the variable to your env before running the script, like this: • Note that Java home should not point to the java executable, but to the jre folder. Applied example: export INSTALL4J_JAVA_HOME_OVERRIDE=/usr/lib/jvm/java-1.8.0-openjdk-amd64 sh illumina _unix_1_13_2_0_35.sh |
Linux | Unsupported version error in log file | If the log file gives the following error "Unsupported major.minor version 52.0", not the right java version is present. The connector makes use of java version 8 or 11. |
Linux | Manage the connector CLI | • Install connector: Chmod +x ./../../Downloads/ illumina _unix_1_13_2_0_35.sh Sudo ./../../Downloads/ illumina _unix_1_13_2_0_35.sh • Start connector with logging (in case log file is not present due to likely the absence of java version 8 or 11) ./illuminaserviceconnector run • Check status of connector from within its location: systemctl status illuminaserviceconnector • Sometimes a connector is not registered as service. This is probably because the connector was installed with local privileges (user) instead of with root. If the user can get root access than they can symlink the service file: ln –s /illuminaserviceconnector /etc/init.d/ Alternatively, the user can start the connector script manually without service scripts from its home folder: ./illuminaserviceconnector start If you work like that, make sure it’s never running twice. You can check whether it’s running with: ./illuminaserviceconnector start If you work like that, make sure it’s never running twice. You can check whether it’s running with: ps aux | grep bluebeeserviceconnector • If connector was started incorrect, start the connector twice afterwards to get it to work. |
| General issue | Solution |
|---|---|
Connector gets connected, but uploads won’t start | Create a new empty folder on your local disk, put a small file in there, and configure this folder as upload folder. • If it works, and your sample files are on a shared drive, have a look at the Shared drives section. • If it works, and your sample files are on your local disk, there are a few possibilities: a) There is an error in how the upload folder name is configured in the platform. b) For big files, or on slow disks, the connector needs quite some time to start the transfer because it needs to calculate a hash to make sure there are no transfer errors. Wait up to 30 minutes, without changing anything to your Connector configuration. • If this doesn’t work, you might have a corporate proxy. Proxy configuration is currently not supported for the connector. |
Upload from shared drive doesn’t work | Follow the guidelines in Shared Drives section. Look in the Connector’s log file whether there’s an error about the directory not being there. • If there is, there are two options: Either there’s an error in the folder name, or there’s a permission problem. In that case, make sure the user running the connector has access without a password being requested to the network share. • If there isn’t, you probably just must wait for quite some time until the consistency check has been done. This check can take quite long on slow disks. |
Data Transfers are slow | Many factors can affect the speed: • Distance from upload location to storage location • Quality of your internet connection • Cable is preferred over wifi • Restrictions for up- and download by the company or the provider. These factors can change every time the customer switches from location e.g. working from home. |
Last updated