diff --git a/content/handling_data/data_transfer/scp.md b/content/handling_data/data_transfer/scp.md index 14c46fb023584df667c8975215d3d6c50a908626..8da4794dda20a9c5482eecb98bc844f6a16c6012 100644 --- a/content/handling_data/data_transfer/scp.md +++ b/content/handling_data/data_transfer/scp.md @@ -44,3 +44,53 @@ on the Crane cluster to your current directory, you would use the command: {{< highlight bash >}} $ scp <user_name>@crane.unl.edu:/work/<group_name>/<user_name>/data.csv ./ {{< /highlight >}} + +### Potential incompatibility with recent versions of scp + +{{% notice note %}} +The scp command when used with shell variable or pathname expanded +source/target arguments, along with certain recursive copy requests, +may return unexpected errors compared with older versions of the command. +{{% /notice %}} + +Beginning with OpenSSH release 8.8, the scp client defaults to using +the SFTP protocol over the legacy SCP protocol. This change in protocol +removes the user's shell from completing environment variable expansion +or wildcard pathname matching on the path component being serviced by +the remote server side of the transfer, as used in the SCP protocol. +This may cause scp transfers to return errors that worked with prior +versions of scp. + +{{< highlight bash >}} +# Recursive copy incompatibility example. +# <source_dir> is missing from the target path resulting in error: +$ scp -r <source_dir> <user_name>@crane.unl.edu:/work/<group_name>/<user_name>/ +scp: realpath /work/<group_name>/<user_name>/<source_dir>: No such file +scp: upload "/work/<group_name>/<user_name>/<source_dir>": path canonicalization failed +scp: failed to upload directory <source_dir> to /work/<group_name>/<user_name>/ +{{< /highlight >}} + + +{{< highlight bash >}} +# Shell environment expansion incompatibility example. +# $WORK is treated as a literal string and not expanded by the remote shell: +$ scp <user_name>@crane.unl.edu:'$WORK/path/to/file' . +scp: $WORK/path/to/file: No such file or directory +{{< /highlight >}} + +To restore the prior shell expansion behavior on the remote server side +of the transfer, add the [-O](https://man.openbsd.org/scp#O) flag in your +scp invocation to use the SCP protocol. + +{{< highlight bash >}} +# Creates <source_dir> and its contents entirely at the target path: +$ scp -O -r <source_dir> <user_name>@crane.unl.edu:/work/<group_name>/<user_name>/ +# Uses the remote shell on crane.unl.edu to expand $WORK to the string +# that is <user_name>'s full "/work/<group_name>/<user_name>" path. +# Note the single quotes to keep the local shell from where scp is invoked +# from attempting to expand the $WORK variable: +$ scp -O <user_name>@crane.unl.edu:'$WORK/path/to/file' . +{{< /highlight >}} + +Details of the change are available at the OpenSSH release 8.8 [Future +deprecation notice](https://www.openssh.com/txt/release-8.8) section.