Data Management Glossary
Symbolic Link
What is a Symbolic Link? What is a symlink?
Symbolic Links, also known as symlinks and symbolic linking, are file-system objects that point toward another file or folder. These links act as shortcuts with advanced properties that allow access to files from locations other than their original place in the folder hierarchy by providing operating systems with instructions on where the “target” file can be found.
For the operating system, the symlink is transparent for many operations and functions in the same manner as the target file or folder would even though it’s only a link that points to the original. For example, if a program needs to be in folder A to run, but you want to store it in folder B instead, the entire A folder could be moved into the B folder with a symbolic link created in folder A which points to folder B. When the program is launched, the operating system would refer to folder A, find the symbolic link to folder B, and run the program from folder B as if it was still in its original place in folder A.
This method is widely used in the storage industry in programs such as OneDrive, Google Drive, and Dropbox to sync files and folders across different platforms of storage or in the cloud.
These types of links began to appear in operating systems in the late 70’s such as RDOS. In modern computing, symbolic links are present in most Unix-like operating systems which are supported by the POSIX standard such as Linux, macOS, and Tru64. This feature was also added to Microsoft Windows starting with Windows Vista.
Symbolic Links vs Hard Links
Both types of symbolic links (also known as symbolic linking) allow seamless and mostly transparent targeting of a file, but they do so in different ways.
Soft links, also referred to as symbolic links by Microsoft, work similarly to a normal shortcut in the sense that they point directly to file or folder itself. These types of links also use less memory overall.
On the other hand, hard links point to the storage space designated to hold the contents of the file or folder.
In this sense, if the location or the name of the file changes, then a soft link would no longer work since it was pointing to the original file itself, but with a hard link, any changes made to the original file or the hard link contents are mirrored by the other because both are pointing to the same location on the storage.
Hard links act as a secondary entrance to the same file or folder which they are linked to, but they can only be used to connect two entities within the same file system, whereas soft links can bridge the gap between different storage devices and file systems.
Hard symbolic links also have more restrictive requirements than soft links:
- Hard links may not be able to link to directories.
- The target file or folder for a hard link must exist.
- Hard links cannot point to targets that are located on different partitions, volumes, or file systems.
Junctions
A Junction is a lesser-used, third type of symbolic link that combines aspects from both hard and soft links. The target file must exist for the junction to be created, but if the target file or folder is erased afterward, the link will still be there but will no longer be functional.
How are Soft and Hard Symbolic Links Commonly Used?
Hard links are used to create “backups” on filesystems without using any additional storage space. This is a benefit as it is often easier to manage a single directory with multiple references pointing to it rather than managing multiple instances of the same directory. If the file or folder is no longer accessible from its original location, then the hard link can be used as a backup to regain access to those files.
The Time Machine feature on macOS uses hard symbolic links to create images to be used for backup.
Soft links are used more heavily to enable access for files and folders on different devices or filesystems. These types of symbolic links are also used in situations where multiple names are being used to link to the same location.
Types of Businesses that Make Use of Symbolic Links
Symbolic links are leveraged in nearly every industry that uses computers, but some industries make use of these links more than others. Below are industries where symbolic links are most commonly used:
- Firms that offer big data management or analytics services
- Engineering & Semiconductor companies
- Financial service firms
- Genomics & healthcare companies
- Higher education institutions
- Media & entertainment organizations
- Oil & gas producers
- Agencies in the public sector
Creating Symbolic Links
The process used to create symbolic links is different on each type of operating system. Below are brief instructions on how a soft or hard link can be set up in Linux and Windows.
How to Create a Soft Link in Linux
To create a soft symbolic link in Linux, the ln command-line utility can be used as such:
ln -s [OPTIONS] FILE LINK
The FILE argument represents the origin of the link. The LINK argument represents the target destination for the soft link.
When the command is successful, there is no output and the command-line will return zero.
How to Create a Hard Link in Linux
For creating hard links in Linux, a similar version of the ln command is used but without the -s:
ln [OPTIONS] FILE LINK
The FILE argument is still the origin location and the LINK argument is still the destination file or directory.
Creating a Windows Soft Link
The mklink command can be used to create soft links in Windows Vista & later through a command prompt or powershell with elevated permissions. By default, this command with no options will produce a soft link.
mklink command:
mklink Link Target
The Link argument is the origin file/directory location and the Target argument represents the intended destination file.
For creating a soft link pointing to a directory, this command is used instead:
mklink /D Link Target
Creating a Windows Hard Link
Similarly to creating a soft link in Windows, the mklink can also be used to create hard links when /H is included as an option as such:
mklink /H Link Target
For creating a junction, the /J option is used instead of /H:
mklink /J Link Target
Komprise Transparent Move Technology (TMT) and Symlinks
The patented Komprise Transparent Move Technology™ (TMT) goes beyond storage-based data tiering to analyze, migrate, tier and replicate data across multi-vendor storage and clouds while enabling native use of the data at each layer. This storage-agnostic data management is possible without disrupting users and without locking data in a proprietary format one vendor’s storage silo.
Komprise TMT uses the standard, built-in feature of Windows, Linux, and Mac symbolic links, which replace a file with a tiny pointer to another location. By using Dynamic Links inside the standard symbolic link, Komprise extends the file system to call these files from the cloud or other storage systems. Dynamic Links dynamically bind a request to the actual data so it can move a file from NFS or SMB to a native cloud object and still provide transparent access from the source.
Read the white paper: Leveraging the Full Power of the Cloud with Komprise Transparent Move Technology.
What is a symbolic link?
A symbolic link, also known as a symlink or soft link, is a file that serves as a reference or pointer to another file or directory in a file system. Unlike a hard link, which points directly to the index node (aka inode – a data structure on a file system on Unix-like operating systems that stores information about a file or a directory), a symbolic link contains a reference to the file’s pathname.
What is a Dynamic Link?
Komprise uses a patented mechanism called Dynamic Links that use standard protocol constructs, eliminate proprietary agents and do not get in the hot data path. Komprise Transparent Move Technology (TMT) uses the standard, built-in feature of Windows, Linux, and Mac called symbolic links which replace a file with a tiny pointer to another location. By using the Komprise Dynamic Link inside the standard symbolic link, the file system is extended to call these files from the cloud or other storage systems. Dynamic Links dynamically bind a request to the actual data so it can move a file from NFS or SMB to a native cloud object and still provide transparent access from the source. As a user if you want to see how this works, click a file that Komprise tiered to the cloud and you get it back instantly. Simply right click the file and you will see that the path points to the Komprise Dynamic Link instead of a file on your computer. Komprise TMT is a scalable, storage-agnostic data movement solution that maintains file and object duality to give you the best of both worlds – transparent data access from the source and native data access outside the data path. In this blog post, Komprise cofounder and CEO noted:
With our patented Dynamic Links, Komprise stays outside the hot data path to deliver a standards-based open data management solution that is resilient and avoids the pitfalls of static stubs or symlinks.
How is symbolic linking different than stubs?
Symbolic linking is a file system feature that allows the creation of references to files or directories, providing a form of aliasing. On the other hand, stubs are placeholders or temporary implementations used in software development, often related to the linking and compilation process, and are replaced with the actual code or functionality at a later stage. The key difference lies in their purposes and the context in which they are used. Symbolic linking and stubs serve different purposes and operate at different levels within a system.
Symbolic Linking:
Symbolic linking, also known as symlink or soft link, is a mechanism in file systems that allows the creation of a special type of file that serves as a symbolic reference or pointer to another file or directory.
- Purpose: Symbolic links, or symlinks, provide a way to create references or pointers to files or directories. They are used to create aliases or shortcuts to other files or directories within the file system.
- Mechanism: A symbolic link is a separate file that contains a path reference to the target file or directory. When the symlink is accessed, the system follows the path reference to the target location.
- Independence: The symlink and the target file or directory have different inodes (index nodes), and they can be located on different file systems. Deleting the symlink does not affect the target, but deleting the target may leave a “dangling” symlink.
- Example: If you have a file named file.txt and create a symlink named link.txt pointing to it, accessing link.txt will effectively access the contents of file.txt.
Note that Komprise uses Dynamic Links. Learn more here.
Stubs:
In software development, a stub is a piece of code or a placeholder that stands in for a more complete or complex implementation. Stubs are used in various stages of the software development life cycle and serve different purposes, primarily related to testing, development, and dependency management.
- Purpose: Stubs are pieces of code or placeholders used in software development and deployment. They are typically temporary implementations or references that are later replaced with the actual code or functionality.
- Mechanism: Stubs can be placeholders or simplified versions of functions or modules. During development, they may serve as stand-ins for more complex or complete implementations until those are available.
- Dependency Resolution: Stubs are often used in the context of linking and compilation. They allow code to be compiled and linked even when some dependencies are not fully implemented.
- Example: In software development, a stub may be used to simulate the behavior of a network interface or a hardware device before the actual device or interface is available. Once the real device is in place, the stub is replaced with the complete implementation.
Traditionally, unstructured data management solutions that move data have relied on one of two approaches:
- They either move the data entirely out of the primary storage, which is undesirable in most scenarios because it creates user friction as users think their files have disappeared.
- If they try to move data transparently, they leave behind a proprietary “stub” file that points to the moved file.
Stubs are problematic for two reasons:
- A stub is proprietary and you need an agent installed on the file storage to detect when the stub is opened. This puts the data movement tool in the hot data path, which is undesirable, as it impacts performance and can become a bottleneck.
- Because a stub statically points to the new location of the file, if a stub is accidentally deleted, then data can get orphaned. Also, a stub can only go to another similar file system, so you cannot bridge file and object for example.
The bottom line is stubs are hard to manage. They need to be backed up, they’re in the data path, they’re limiting, and they are risky because they are a single point of failure. Komprise eliminates these issues by using a patented mechanism called Dynamic Links that uses standard protocol constructs and eliminate proprietary agents without getting in the hot data path.