Tentative/msys2 3.6.0 #260
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jeremyd2019
force-pushed
the
tentative/msys2-3.6.0
branch
from
a4d28ed to
0dd972a
Compare
|
Interesting test results:
|
|
@dscho fyi I added a new |
|
Should we think about pulling in git-for-windows/msys2-runtime@f92c1e9? I tried to point upstream cygwin at it when somebody reported an issue with scp, but it doesn't seem to have gotten any traction with them. |
jeremyd2019
force-pushed
the
tentative/msys2-3.6.0
branch
from
0dd972a to
86ecfd1
Compare
|
I came up with a git range-diff cygwin-A.B.C..msys2-A.B.C msys2-X.Y.Z..tentative/msys2-X.Y.Z | sed '/^ /,/^\([^ ]\| [^ ]\| [^ ]\| [^ ]\)/ {
/^\([^ ]\| [^ ]\| [^ ]\| [^ ]\)/ {x;s/^\n//;i ```diff
p;x;i ```
b}; s/^ //;H;d;b}' |
jeremyd2019
force-pushed
the
tentative/msys2-3.6.0
branch
from
86ecfd1 to
c665bc8
Compare
This has been applied upstream now. |
There was a bug in the sed script, now: /^ /,/^\([^ ]\| [^ ]\| [^ ]\| [^ ]\)/ {
/^\([^ ]\| [^ ]\| [^ ]\| [^ ]\)/ {x;s/^\n//;i ```diff
p;z;x;i ```
b}; s/^ //;H;d} |
jeremyd2019
force-pushed
the
tentative/msys2-3.6.0
branch
from
c665bc8 to
8a00a74
Compare
Another bug, now /^ /,/^ \{0,3\}\([^ ]\|$\)/ {
s/^ //;T2;H;${z;b2};d};b;:2;x;s/^\n//;i ```diff
p;z;x;i ``` |
jeremyd2019
force-pushed
the
tentative/msys2-3.6.0
branch
from
8a00a74 to
1bc1047
Compare
|
In today's rebase, Corinna fixed the warnings, so I got to drop a couple of commits here disabling warnings-as-errors |
jeremyd2019
force-pushed
the
tentative/msys2-3.6.0
branch
from
1bc1047 to
cd1b13e
Compare
How have I missed the For the record, I am piping range-diff output to this script for years: #!/bin/sh
sed -e '/^ \{0,3\}\(-\|[1-9][0-9]*\):/{a\
\
``````diff
s/^ */* /;1b;i\
``````\
}' -e 's/^ / /' -e '$a\
``````' | sed -e '1i\
\
' -e '/^$/{N;/^\n ``````diff/{N;/diff\n ``````/{$d;N;d}}}'It ain't pretty, but it works. |
That's another GNU extension, easily avoided. I thought about replacing it with
I'll have to think about if I prefer the bullets and indented diff blocks. Shouldn't be too hard to add to my version. I assume the 6x backticks an artifact of putting it in the ```shell block? |
Something like /^ /,/^ \{0,3\}\([^ ]\|$\)/ {
s/^ / /;T2;H;${z;b2};d};b3;:2;x;s/^\n//;i\
```diff
p;z;x;i\
```
:3;/^ \{0,3\}\(-\|[1-9][0-9]*\):/{s/ */* /}wiki updated |
|
for fun, here's a version that also works on Solaris 9's sed and OpenBSD's sed (so probably posix-compliant) #!/usr/bin/sed -f
/^ /,/^ \{0,3\}[^ ]/ {
s/^ / /
t1
b2
:1
H
${
s/.*//
b2
}
d
}
b3
:2
x
s/^\n//
i\
```diff
p
s/.*//
x
i\
```
:3
/^ \{0,3\}[-1-9][0-9]*:/ {
s/ */* /
}I had to loosen the patterns a bit because of lack of posix support for alternation |
jeremyd2019
force-pushed
the
tentative/msys2-3.6.0
branch
from
cd1b13e to
8ca2595
Compare
Cygwin's speclib doesn't handle dashes or dots. However, we are about to rename the output file name from `cygwin1.dll` to `msys-2.0.dll`. Let's preemptively fix up all the import libraries that would link against `msys_2_0.dll` to correctly link against `msys-2.0.dll` instead.
…ent variables to Windows form for native Win32 applications.
…t without ACLs. - Can read /etc/fstab with short mount point format.
The new `winsymlinks` mode `deepcopy` (which is made the default) lets calls to `symlink()` create (deep) copies of the source file/directory. This is necessary because unlike Cygwin, MSYS2 does not try to be its own little ecosystem that lives its life separate from regular Win32 programs: the latter have _no idea_ about Cygwin-emulated symbolic links (i.e. system files whose contents start with `!<symlink>\xff\xfe` and the remainder consists of the NUL-terminated, UTF-16LE-encoded symlink target). To support Cygwin-style symlinks, the new mode `sysfile` is introduced. Co-authored-by: Johannes Schindelin <[email protected]> Co-authored-by: Jeremy Drake <[email protected]>
With MSys1, it was necessary to set the TERM variable to "msys". To allow for a smooth transition from MSys1 to MSys2, let's simply handle TERM=msys as if the user had not specified TERM at all and wanted us to use our preferred TERM value.
Strace is a Windows program so MSYS2 will convert all arguments and environment vars and that makes debugging msys2 software with strace very tricky.
Commit message for this code was: * strace.cc (create_child): Set CYGWIN=noglob when starting new process so that Cygwin will leave already-parsed the command line alonw." I can see no reason for it and it badly breaks the ability to use strace.exe to investigate calling a Cygwin program from a Windows program, for example: strace mingw32-make.exe .. where mingw32-make.exe finds sh.exe and uses it as the shell. The reason it badly breaks this use-case is because dcrt0.cc depends on globbing to happen to parse commandlines from Windows programs; irrespective of whether they contain any glob patterns or not. See quoted () comment: "This must have been run from a Windows shell, so preserve quotes for globify to play with later."
In theory this doesn't make a difference because posix_to_win32_path() is only called with rooted/absolute paths, but as pointed out in #103 PC_NOFULL will preserve the trailing slash of unix paths (for some reason). See "cygpath -m /bin/" (preserved) vs "cygpath -am /bin/" (dropped) One use case where we need to trailing slashes to be preserved is the GCC build system: https://github.com/gcc-mirror/gcc/blob/6d82e0fea5f988e829912a/gcc/Makefile.in#L2314 The Makefile appends a slash to the prefixes and the C code doing relocation will treat the path as a directory if there is a trailing slash. See msys2/MINGW-packages#14173 for details. With this change all our MSYS2 path_conv tests pass again.
When calling windows native apps from MSYS2, the runtime tries to convert commandline arguments by a specific set of rules. This idea was inherited from the MSys/MinGW project (which is now seemingly stale, yet must be credited with championing this useful feature, see MinGW wiki https://web.archive.org/web/20201112005258/http://www.mingw.org/wiki/Posix_path_conversion). If the user does not want that behavior on a big scale, e.g. inside a Bash script, with the changes introduced in this commit, the user can now set the the environment variable `MSYS_NO_PATHCONV` when calling native windows commands. This is a feature that has been introduced in Git for Windows via git-for-windows/msys2-runtime#11 and it predates support for the `MSYS2_ENV_CONV_EXCL` and `MSYS2_ARG_CONV_EXCL` environment variables in the MSYS2 runtime; Many users find the simplicity of `MSYS_NO_PATHCONV` appealing. So let's teach MSYS2 proper this simple trick that still allows using the sophisticated `MSYS2_*_CONV_EXCL` facilities but also offers a convenient catch-all "just don't convert anything" knob. Signed-off-by: 마누엘 <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Otherwise if globbing is allowed and we get called from a Windows program, build_argv thinks we've been called from a Cygwin program.
…spec Reverts 25ba8f3. I can't figure out what the intention was. I'm sure I'll find out soon enough when everything breaks. This change means that input of: '"C:/test.exe SOME_VAR=\"literal quotes\""' becomes: 'C:/test.exe SOME_VAR="literal quotes"' instead of: 'C:/test.exe SOME_VAR=\literal quotes\' .. which is at least consistent with the result for: '"no_drive_or_colon SOME_VAR=\"literal quotes\""' The old result of course resulted in the quoted string being split into two arguments at the space which is clearly not intended. I *guess* backslashes in dos paths may have been the issue here? If so I don't care since we should not use them, ever, esp. not at the expense of sensible forward-slash-containing input.
Works very much like MSYS2_ARG_CONV_EXCL. In fact it uses the same function, arg_heuristic_with_exclusions (). Also refactors parsing the env. variables to use new function, string_split_delimited (). The env. that is searched through is the merged (POSIX + Windows) one. It remains to be seen if this should be made an option or not. This feature was prompted because the R language (Windows exe) calls bash to run configure.win, which then calls back into R to read its config variables (LOCAL_SOFT) and when this happens, msys2-runtime converts R_ARCH from "/x64" to an absolute Windows path and appends it to another absolute path, R_HOME, forming an invalid path.
It is simply the negation of `disable_pcon`, i.e. `MSYS=enable_pcon` is equivalent to `MSYS=nodisable_pcon` (the former is slightly more intuitive than the latter) and likewise `MSYS=noenable_pcon` is equivalent to `MSYS=disable_pcon` (here, the latter is definitely more intuitive than the former). This is needed because we just demoted the pseudo console feature to be opt-in instead of opt-out, and it would be awkward to recommend to users to use "nodisable_pcon"... "nodisable" is not even a verb. Signed-off-by: Johannes Schindelin <[email protected]>
We mount /usr/bin to /bin, but in a chroot this is broken and we have no /bin, so try to use the real path. chroot is used by pacman to run install scripts when called with --root and this broke programs in install scripts calling popen() (install-info from texinfo for example) There are more paths hardcoded to /bin in cygwin which might also be broken in this scenario, so this maybe should be extended to all of them.
It does not work at all. For example, `rpm -E %fedora` says that there should be version 33 of rpmsphere at https://github.com/rpmsphere/noarch/tree/master/r, but there is only version 32. Another thing that is broken: Cygwin now assumes that a recent mingw-w64-headers version is available, but Fedora apparently only offers v7.0.0, which is definitely too old to accommodate for the expectation of cygwin/cygwin@c1f7c4d1b6d7. Signed-off-by: Johannes Schindelin <[email protected]>
Build with --disable-dependency-tracking because we only build once and this saves 3-4 minutes in CI.
This will help us by automating an otherwise tedious task. Signed-off-by: Johannes Schindelin <[email protected]>
In the Cygwin project, it was decided that the command-line of Cygwin processes, as shown in the output of `wmic process list`, would suffer from being truncated to 32k (and is transmitted to the child process via a different mechanism, anyway), and therefore only the absolute path of the executable is shown by default. Users who would like to see the full command-line (even if it is truncated) are expected to set `CYGWIN=wincmdln` (or, in MSYS2's case, `MSYS=wincmdln`). Seeing as MSYS2 tries to integrate much better with the surrounding Win32 ecosystem than Cygwin, it makes sense to turn this on by default. Users who wish to suppress it can still set `MSYS=nowincmdln`. Signed-off-by: Johannes Schindelin <[email protected]>
In particular, we are interested in the address of the CtrlRoutine and the ExitProcess functions. Since kernel32.dll is loaded first thing, the addresses will be the same for all processes (matching the CPU architecture, of course). This will help us with emulating SIGINT properly (by not sending signals to *all* processes attached to the same Console, as GenerateConsoleCtrlEvent() would do). Co-authored-by: Naveen M K <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
This patch is heavily inspired by the Git for Windows' strategy in handling Ctrl+C. When a process is terminated via TerminateProcess(), it has no chance to do anything in the way of cleaning up. This is particularly noticeable when a lengthy Git for Windows process tries to update Git's index file and leaves behind an index.lock file. Git's idea is to remove the stale index.lock file in that case, using the signal and atexit handlers available in Linux. But those signal handlers never run. Note: this is not an issue for MSYS2 processes because MSYS2 emulates Unix' signal system accurately, both for the process sending the kill signal and the process receiving it. Win32 processes do not have such a signal handler, though, instead MSYS2 shuts them down via `TerminateProcess()`. For a while, Git for Windows tried to use a gentler method, described in the Dr Dobb's article "A Safer Alternative to TerminateProcess()" by Andrew Tucker (July 1, 1999), http://www.drdobbs.com/a-safer-alternative-to-terminateprocess/184416547 Essentially, we injected a new thread into the running process that does nothing else than running the ExitProcess() function. However, this was still not in line with the way CMD handles Ctrl+C: it gives processes a chance to do something upon Ctrl+C by calling SetConsoleCtrlHandler(), and ExitProcess() simply never calls that handler. So for a while we tried to handle SIGINT/SIGTERM by attaching to the console of the command to interrupt, and generating the very same event as CMD does via GenerateConsoleCtrlEvent(). This method *still* was not correct, though, as it would interrupt *every* process attached to that Console, not just the process (and its children) that we wanted to signal. A symptom was that hitting Ctrl+C while `git log` was shown in the pager would interrupt *the pager*. The method we settled on is to emulate what GenerateConsoleCtrlEvent() does, but on a process by process basis: inject a remote thread and call the (private) function kernel32!CtrlRoutine. To obtain said function's address, we use the dbghelp API to generate a stack trace from a handler configured via SetConsoleCtrlHandler() and triggered via GenerateConsoleCtrlEvent(). To avoid killing each and all processes attached to the same Console as the MSYS2 runtime, we modify the cygwin-console-helper to optionally print the address of kernel32!CtrlRoutine to stdout, and then spawn it with a new Console. Note that this also opens the door to handling 32-bit process from a 64-bit MSYS2 runtime and vice versa, by letting the MSYS2 runtime look for the cygwin-console-helper.exe of the "other architecture" in a specific place (we choose /usr/libexec/, as it seems to be the convention for helper .exe files that are not intended for public consumption). The 32-bit helper implicitly links to libgcc_s_dw2.dll and libwinpthread-1.dll, so to avoid cluttering /usr/libexec/, we look for the helped of the "other" architecture in the corresponding mingw32/ or mingw64/ subdirectory. Among other bugs, this strategy to handle Ctrl+C fixes the MSYS2 side of the bug where interrupting `git clone https://...` would send the spawned-off `git remote-https` process into the background instead of interrupting it, i.e. the clone would continue and its progress would be reported mercilessly to the console window without the user being able to do anything about it (short of firing up the task manager and killing the appropriate task manually). Note that this special-handling is only necessary when *MSYS2* handles the Ctrl+C event, e.g. when interrupting a process started from within MinTTY or any other non-cmd-based terminal emulator. If the process was started from within `cmd.exe`'s terminal window, child processes are already killed appropriately upon Ctrl+C, by `cmd.exe` itself. Also, we can't trust the processes to end it's subprocesses upon receiving Ctrl+C. For example, `pip.exe` from `python-pip` doesn't kill the python it lauches (it tries to but fails), and I noticed that in cmd it kills python also correctly, which mean we should kill all the process using `exit_process_tree`. Co-authored-by: Naveen M K <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
This change is the equivalent to the change to the Ctrl+C handling we just made. Co-authored-by: Naveen M K <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
This code has been causing issues with SUBST and mapped network drives, so add an option (defaulted to on) which can be used to disable it where needed. MSYS=nonativeinnerlinks
The MSYS2 packages lack the infrastructure to build those. Signed-off-by: Johannes Schindelin <[email protected]>
Before symlinking libg.a, we need the symlink source `libmsys-2.0.a`: in MSYS2, we copy by default (if we were creating Unix-style symlinks, the target would not have to exist before symlinking, but when copying we do need the source _right away_). Signed-off-by: Johannes Schindelin <[email protected]>
When calling a non-MSys2 binary, all of the environment is converted from POSIX to Win32, including the SHELL environment variable. In Git for Windows, for example, `SHELL=/usr/bin/bash` is converted to `SHELL=C:\Program Files\Git\usr\bin\bash.exe` when calling the `git.exe` binary. This is appropriate because non-MSys2 binaries would not handle POSIX paths correctly. Under certain circumstances, however, `git.exe` calls an *MSys2* binary in turn, such as `git config --edit` calling `vim.exe` unless Git is configured to use another editor specifically. Now, when this "improved vi" calls shell commands, it uses that $SHELL variable *without quoting*, resulting in a nasty error: C:\Program: No such file or directory Many other programs behave in the same manner, assuming that $SHELL does not contain spaces and hence needs no quoting, unfortunately including some of Git's own scripts. Therefore let's make sure that $SHELL gets "posified" again when entering MSys2 programs. Earlier attempts by Git for Windows contributors claimed that adding `SHELL` to the `conv_envvars` array does not have the intended effect. These reports just missed that the `conv_start_chars` array (which makes the code more performant) needs to be adjusted, too. Note that we set the `immediate` flag to `true` so that the environment variable is set immediately by the MSys2 runtime, i.e. not only spawned processes will see the POSIX-ified `SHELL` variable, but the MSys2 runtime *itself*, too. This fixes git-for-windows/git#542, git-for-windows/git#498, and git-for-windows/git#468. Signed-off-by: Johannes Schindelin <[email protected]>
MSYS2 recently introduced that hack where the ORIGINAL_PATH variable is set to the original PATH value in /etc/profile, unless previously set. In Git for Windows' default mode, that ORIGINAL_PATH value is the used to define the PATH variable explicitly. So far so good. The problem: when calling from inside an MSYS2 process (such as Bash) a MINGW executable (such as git.exe) that then calls another MSYS2 executable (such as bash.exe), that latter call will try to re-convert ORIGINAL_PATH after the previous call converted ORIGINAL_PATH from POSIX to Windows paths. And this conversion may very well fail, e.g. when the path list contains mixed semicolons and colons. So let's just *force* the MSYS2 runtime to handle ORIGINAL_PATH in the same way as the PATH variable (which conversion works, as we know). Signed-off-by: Johannes Schindelin <[email protected]>
We are currently trying to move our cygwin build environment closer to cygwin and some autotools/bash based build systems call "uname -s" to figure out the OS and in many cases only handle the cygwin case, so we have to patch them. With this instead of patching we can set MSYSTEM=CYGWIN and change uname output that way. The next step would be to always output CYGWIN in an msys env by default, but for now this allows us to get rid of all the patches without affecting users.
There is a difference between an empty value and an unset environment variable. We should not confuse both; If the user wants to unset an environment variable, they can certainly do so (unsetenv(3), or in the shell: 'unset ABC'). This fixes Git's t3301-notes.sh, which overrides environment variables with empty values. Signed-off-by: Johannes Schindelin <[email protected]>
We just disabled the code that skips environment variables whose values are empty. However, this code was introduced a long time ago into Cygwin in d6b1ac7 (* environ.cc (build_env): Don't put an empty environment variable into the environment. Optimize use of "len". * errno.cc (ERROR_MORE_DATA): Translate to EMSGSIZE rather than EAGAIN., 2006-09-07), seemingly without any complaints. Meaning: There might very well be use cases out there where it makes sense to skip empty-valued environment variables. Therefore, it seems like a good idea to have a "knob" to turn it back on. With this commit, we introduce such a knob: by setting `noemptyenvvalues` the `MSYS` variable (or appending it if that variable is already set), users can tell the MSYS2 runtime to behave just like in the olden times. Signed-off-by: Johannes Schindelin <[email protected]>
With this commit, you can call MSYS=noemptyenvvalues my-command and it does what is expected: to pass no empty-valued environment variables to `my-command`. Signed-off-by: Johannes Schindelin <[email protected]>
This reverts commit 943433b. This seems to fix fork errors under Docker, see https://cygwin.com/pipermail/cygwin/2022-December/252711.html
It frequently leads to problems when trying, say, to call from MSYS2's
Bash into Cygwin's or Git for Windows', merely because sharing that data
is pretty finicky.
For example, using the MSYS2' Bash using the MSYS2 runtime version that
is current at time of writing, trying to call Cygwin's programs fails
in manners like this:
$ /c/cygwin64/bin/uname -r
0 [main] uname (9540) child_copy: cygheap read copy failed, 0x800000000..0x800010BE0, done 0, windows pid 9540, Win32 error 6
680 [main] uname 880 C:\cygwin64\bin\uname.exe: *** fatal error - couldn't create signal pipe, Win32 error 5
with the rather misleading exit code 127 (a code which is reserved to
indicate that a command was not found).
Let's just treat the MSYS2 runtime and the Cygwin runtime as completely
incompatible with one another, by virtue of using a different
magic constant than merely `CHILD_INFO_MAGIC`.
By using the msys2-runtime commit to modify that magic constant, we can
even spawn programs using a different MSYS2 runtime (such as Git for
Windows') because the commit serves as the tell-tale whether two MSYS2
runtime versions are compatible with each other. To support building in
the MSYS2-packages repository (where we do not check out the
`msys2-runtime` but instead check out Cygwin and apply patches on top),
let's accept a hard-coded commit hash as `./configure` option.
One consequence is that spawned MSYS processes using a different MSYS2
runtime will not be visible as such to the parent process, i.e. they
cannot share any resources such as pseudo terminals. But that's okay,
they are simply treated as if they were regular Win32 programs.
Note: We have to use a very rare form of encoding the brackets in the
`expr` calls: quadrigraphs (for a thorough explanation, see
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.70/html_node/Quadrigraphs.html#Quadrigraphs).
This is necessary because it is apparently impossible to encode brackets
in `configure.ac` files otherwise.
Signed-off-by: Johannes Schindelin <[email protected]>
Having just Cygwin's version in the output of `uname` is not helpful, as both MSYS2 as well as Git for Windows release intermediate versions of the MSYS2 runtime much more often than Cygwin runtime versions are released. Signed-off-by: Johannes Schindelin <[email protected]>
Signed-off-by: Jeremy Drake <[email protected]>
jeremyd2019
force-pushed
the
tentative/msys2-3.6.0
branch
from
8ca2595 to
386372e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
context: cygwin-3.5.7...cygwin-3.6.0
git range-diff --creation-factor=80 cygwin-3.5.7..msys2-3.5.7 msys2-3.6.0..tentative/msys2-3.6.0
@@ winsup/cygwin/uname.cc: uname_x (struct utsname *name) memset (name, 0, sizeof (*name)); /* sysname */ -- __small_sprintf (name->sysname, "CYGWIN_%s-%u", +- n = __small_sprintf (name->sysname, "CYGWIN_%s-%u", + char* msystem = getenv("MSYSTEM"); + const char* msystem_sysname = "MSYS"; + if (msystem != NULL && *msystem && strcmp(msystem, "MSYS") != 0) + msystem_sysname = (strstr(msystem, "32") != NULL) ? "MINGW32" : "MINGW64";; -+ __small_sprintf (name->sysname, "%s_%s-%u", -+ msystem_sysname, - wincap.osname (), wincap.build_number ()); - /* nodename */ - memset (buf, 0, sizeof buf); ++ n = __small_sprintf (name->sysname, "%s_%s-%u", ++ msystem_sysname, + wincap.osname (), wincap.build_number ()); + if (wincap.host_machine () != wincap.cygwin_machine ()) + { @@ winsup/cygwin/uname.cc: uname_x (struct utsname *name) /* Old entrypoint for applications up to API 334 */ struct old_utsname@@ Commit message To support Cygwin-style symlinks, the new mode `sysfile` is introduced. Co-authored-by: Johannes Schindelin <[email protected]> + Co-authored-by: Jeremy Drake <[email protected]> ## winsup/cygwin/environ.cc ## @@ winsup/cygwin/environ.cc: set_winsymlinks (const char *buf) @@ winsup/cygwin/path.cc: conv_path_list (const char *src, char *dst, size_t size, /********************** Symbolic Link Support **************************/ ++static int ++recursiveCopyCheckSymlink(PUNICODE_STRING src, bool& isdirlink) ++{ ++ path_conv pc (src, PC_SYM_NOFOLLOW|PC_SYM_NOFOLLOW_REP); ++ if (pc.error) ++ { ++ set_errno (pc.error); ++ return -1; ++ } ++ isdirlink = pc.issymlink (); ++ return 0; ++} ++ +/* + Create a deep copy of src as dst, while avoiding descending in origpath. +*/ +static int -+recursiveCopy (char * src, char * dst, const char * origpath) ++recursiveCopy (PUNICODE_STRING src, PUNICODE_STRING dst, USHORT origsrclen, ++ USHORT origdstlen, PWIN32_FIND_DATAW dHfile = NULL) +{ -+ WIN32_FIND_DATA dHfile; + HANDLE dH = INVALID_HANDLE_VALUE; -+ BOOL findfiles; -+ int srcpos = strlen (src); -+ int dstpos = strlen (dst); ++ NTSTATUS status; ++ int srcpos = src->Length; ++ int dstpos = dst->Length; + int res = -1; ++ bool freedHfile = false; + -+ debug_printf("recursiveCopy (%s, %s)", src, dst); ++ if (!dHfile) ++ { ++ dHfile = (PWIN32_FIND_DATAW) cmalloc_abort (HEAP_STR, sizeof (*dHfile)); ++ freedHfile = true; ++ } ++ ++ debug_printf ("recursiveCopy (%S, %S)", src, dst); + + /* Create the destination directory */ -+ if (!CreateDirectoryEx (src, dst, NULL)) ++ if (!CreateDirectoryExW (src->Buffer, dst->Buffer, NULL)) + { -+ debug_printf("CreateDirectoryEx(%s, %s, 0) failed", src, dst); ++ debug_printf ("CreateDirectoryExW(%S, %S, 0) failed", src, dst); + __seterrno (); + goto done; + } + /* Descend into the source directory */ -+ if (srcpos + 2 >= MAX_PATH || dstpos + 1 >= MAX_PATH) ++ if (src->Buffer[(src->Length - 1) / sizeof (WCHAR)] != L'\\') ++ { ++ status = RtlAppendUnicodeToString (src, L"\\*"); ++ } ++ else ++ { ++ status = RtlAppendUnicodeToString (src, L"*"); ++ srcpos -= sizeof (WCHAR); ++ } ++ if (!NT_SUCCESS (status)) ++ { ++ __seterrno_from_nt_status (status); ++ goto done; ++ } ++ if (dst->Buffer[(dst->Length - 1) / sizeof (WCHAR)] != L'\\') ++ status = RtlAppendUnicodeToString (dst, L"\\"); ++ else ++ dstpos -= sizeof (WCHAR); ++ if (!NT_SUCCESS (status)) + { -+ set_errno (ENAMETOOLONG); ++ __seterrno_from_nt_status (status); + goto done; + } -+ strcat (src, "\\*"); -+ strcat (dst, "\\"); -+ dH = FindFirstFile (src, &dHfile); -+ debug_printf("dHfile(1): %s", dHfile.cFileName); -+ findfiles = FindNextFile (dH, &dHfile); -+ debug_printf("dHfile(2): %s", dHfile.cFileName); -+ findfiles = FindNextFile (dH, &dHfile); -+ while (findfiles) ++ ++ dH = FindFirstFileExW (src->Buffer, FindExInfoBasic, dHfile, ++ FindExSearchNameMatch, NULL, ++ FIND_FIRST_EX_LARGE_FETCH); ++ if (dH == INVALID_HANDLE_VALUE) + { ++ __seterrno (); ++ goto done; ++ } ++ ++ do ++ { ++ bool isdirlink = false; ++ debug_printf ("dHfile: %W", dHfile->cFileName); ++ if (dHfile->cFileName[0] == L'.' && ++ (!dHfile->cFileName[1] || ++ (dHfile->cFileName[1] == L'.' && !dHfile->cFileName[2]))) ++ continue; + /* Append the directory item filename to both source and destination */ -+ int filelen = strlen (dHfile.cFileName); -+ debug_printf("dHfile(3): %s", dHfile.cFileName); -+ if (srcpos + 1 + filelen >= MAX_PATH || -+ dstpos + 1 + filelen >= MAX_PATH) -+ { -+ set_errno (ENAMETOOLONG); -+ goto done; -+ } -+ strcpy (&src[srcpos+1], dHfile.cFileName); -+ strcpy (&dst[dstpos+1], dHfile.cFileName); -+ debug_printf("%s -> %s", src, dst); -+ if (dHfile.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) ++ src->Length = srcpos + sizeof (WCHAR); ++ dst->Length = dstpos + sizeof (WCHAR); ++ status = RtlAppendUnicodeToString (src, dHfile->cFileName); ++ if (!NT_SUCCESS (status)) ++ { ++ __seterrno_from_nt_status (status); ++ goto done; ++ } ++ status = RtlAppendUnicodeToString (dst, dHfile->cFileName); ++ if (!NT_SUCCESS (status)) ++ { ++ __seterrno_from_nt_status (status); ++ goto done; ++ } ++ debug_printf ("%S -> %S", src, dst); ++ if ((dHfile->dwFileAttributes & ++ (FILE_ATTRIBUTE_DIRECTORY|FILE_ATTRIBUTE_REPARSE_POINT)) == ++ (FILE_ATTRIBUTE_DIRECTORY|FILE_ATTRIBUTE_REPARSE_POINT)) ++ { ++ /* I was really hoping to avoid using path_conv in the recursion, ++ but maybe putting it in its own function will prevent it from ++ taking up space in the stack frame */ ++ if (recursiveCopyCheckSymlink (src, isdirlink)) ++ goto done; ++ } ++ if (isdirlink) + { -+ /* Recurse into the child directory */ -+ debug_printf("%s <-> %s", src, origpath); -+ if (strcmp (src, origpath)) // avoids endless recursion -+ if (recursiveCopy (src, dst, origpath)) -+ goto done; -+ } ++ /* CreateDirectoryEx seems to "copy" directory reparse points, which ++ CopyFileEx can only do with a flag introduced in 19041. */ ++ if (!CreateDirectoryExW (src->Buffer, dst->Buffer, NULL)) ++ { ++ debug_printf ("CreateDirectoryExW(%S, %S, 0) failed", src, dst); ++ __seterrno (); ++ goto done; ++ } ++ } ++ else if (dHfile->dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) ++ { ++ /* Recurse into the child directory */ ++ /* avoids endless recursion */ ++ if (src->Length <= origsrclen || ++ !wcsncmp (src->Buffer, dst->Buffer, origdstlen / sizeof (WCHAR))) ++ { ++ set_errno (ELOOP); ++ goto done; ++ } ++ if (recursiveCopy (src, dst, origsrclen, origdstlen, dHfile)) ++ goto done; ++ } + else -+ { -+ /* Just copy the file */ -+ if (!CopyFile (src, dst, FALSE)) -+ { -+ __seterrno (); -+ goto done; -+ } -+ } -+ findfiles = FindNextFile (dH, &dHfile); ++ { ++ /* Just copy the file */ ++ if (!CopyFileExW (src->Buffer, dst->Buffer, NULL, NULL, NULL, ++ COPY_FILE_COPY_SYMLINK)) ++ { ++ __seterrno (); ++ goto done; ++ } ++ } + } ++ while (FindNextFileW (dH, dHfile)); ++ + if (GetLastError() != ERROR_NO_MORE_FILES) + { + __seterrno (); @@ winsup/cygwin/path.cc: conv_path_list (const char *src, char *dst, size_t size, + if (dH != INVALID_HANDLE_VALUE) + FindClose (dH); + ++ if (freedHfile) ++ cfree (dHfile); ++ + return res; +} + /* Create a symlink from FROMPATH to TOPATH. */ extern "C" int -@@ winsup/cygwin/path.cc: symlink_worker (const char *oldpath, path_conv &win32_newpath, bool isdevice) - } - else /* wsym_type == WSYM_sysfile */ - { -+ if (wsym_type == WSYM_deepcopy) +@@ winsup/cygwin/path.cc: symlink_wsl (const char *oldpath, path_conv &win32_newpath) + return 0; + } + ++int ++symlink_deepcopy (const char *oldpath, path_conv &win32_newpath) ++{ ++ tmp_pathbuf tp; ++ path_conv win32_oldpath; ++ ++ /* **BEGIN** replace this with ++ resolve_symlink_target (oldpath, win32_newpath. win32_oldpath); ++ when rebasing over 5a706ff0fceb83fd1fe7f072fc28a741fdde65f2 ++ (probably Cygwin 3.6) */ ++ /* The symlink target is relative to the directory in which the ++ symlink gets created, not relative to the cwd. Therefore we ++ have to mangle the path quite a bit before calling path_conv.*/ ++ if (isabspath (oldpath)) ++ win32_oldpath.check (oldpath, PC_SYM_NOFOLLOW, stat_suffixes); ++ else ++ { ++ size_t len = strrchr (win32_newpath.get_posix (), '/') ++ - win32_newpath.get_posix () + 1; ++ char *absoldpath = tp.t_get (); ++ stpcpy (stpncpy (absoldpath, win32_newpath.get_posix (), len), ++ oldpath); ++ win32_oldpath.check (absoldpath, PC_SYM_NOFOLLOW, stat_suffixes); ++ } ++ /* **END** */ ++ if (win32_oldpath.error) ++ { ++ set_errno (win32_oldpath.error); ++ return -1; ++ } ++ if (win32_oldpath.isspecial ()) ++ return -2; ++ ++ /* MSYS copy file instead make symlink */ ++ /* As a MSYS limitation, the source path must exist. */ ++ if (!win32_oldpath.exists ()) ++ { ++ set_errno (ENOENT); ++ return -1; ++ } ++ ++ PUNICODE_STRING w_oldpath = win32_oldpath.get_nt_native_path (); ++ PUNICODE_STRING w_newpath = win32_newpath.get_nt_native_path (); ++ if (w_oldpath->Buffer[1] == L'?') ++ w_oldpath->Buffer[1] = L'\\'; ++ if (w_newpath->Buffer[1] == L'?') ++ w_newpath->Buffer[1] = L'\\'; ++ if (win32_oldpath.isdir ()) ++ { ++ /* we need a larger UNICODE_STRING MaximumLength than ++ get_nt_native_path allocates for the recursive copy */ ++ UNICODE_STRING u_oldpath, u_newpath; ++ RtlCopyUnicodeString (tp.u_get (&u_oldpath), w_oldpath); ++ RtlCopyUnicodeString (tp.u_get (&u_newpath), w_newpath); ++ return recursiveCopy (&u_oldpath, &u_newpath, ++ u_oldpath.Length, u_newpath.Length); ++ } ++ else ++ { ++ bool isdirlink = false; ++ if (win32_oldpath.issymlink () && ++ win32_oldpath.is_known_reparse_point ()) ++ { ++ /* Is there a better way to know this? */ ++ DWORD attr = getfileattr (win32_oldpath.get_win32 (), ++ !!win32_oldpath.objcaseinsensitive ()); ++ if (attr == INVALID_FILE_ATTRIBUTES) + { -+ path_conv src_path; -+ src_path.check (oldpath, PC_SYM_NOFOLLOW, stat_suffixes); -+ if (src_path.error) -+ { -+ set_errno (src_path.error); -+ __leave; -+ } -+ if (!src_path.isspecial ()) -+ { -+ /* MSYS copy file instead make symlink */ -+ -+ char * real_oldpath; -+ if (isabspath (oldpath)) -+ strcpy (real_oldpath = tp.c_get (), oldpath); -+ else -+ /* Find the real source path, relative -+ to the directory of the destination */ -+ { -+ /* Determine the character position of the last path component */ -+ const char *newpath = win32_newpath.get_posix(); -+ int pos = strlen (newpath); -+ while (--pos >= 0) -+ if (isdirsep (newpath[pos])) -+ break; -+ /* Append the source path to the directory -+ component of the destination */ -+ if (pos+1+strlen(oldpath) >= MAX_PATH) -+ { -+ set_errno(ENAMETOOLONG); -+ __leave; -+ } -+ strcpy (real_oldpath = tp.c_get (), newpath); -+ strcpy (&real_oldpath[pos+1], oldpath); -+ } -+ -+ /* As a MSYS limitation, the source path must exist. */ -+ path_conv win32_oldpath; -+ win32_oldpath.check (real_oldpath, PC_SYM_NOFOLLOW, stat_suffixes); -+ if (!win32_oldpath.exists ()) -+ { -+ set_errno (ENOENT); -+ __leave; -+ } -+ -+ char *w_newpath; -+ char *w_oldpath; -+ stpcpy (w_newpath = tp.c_get (), win32_newpath.get_win32()); -+ stpcpy (w_oldpath = tp.c_get (), win32_oldpath.get_win32()); -+ if (win32_oldpath.isdir()) -+ { -+ char *origpath; -+ strcpy (origpath = tp.c_get (), w_oldpath); -+ res = recursiveCopy (w_oldpath, w_newpath, origpath); -+ } -+ else -+ { -+ if (!CopyFile (w_oldpath, w_newpath, FALSE)) -+ { -+ __seterrno (); -+ } -+ else -+ { -+ res = 0; -+ } -+ } -+ __leave; -+ } ++ __seterrno (); ++ return -1; + } ++ isdirlink = attr & FILE_ATTRIBUTE_DIRECTORY; ++ } ++ if (isdirlink) ++ { ++ /* CreateDirectoryEx seems to "copy" directory reparse points, which ++ CopyFileEx can only do with a flag introduced in 19041. */ ++ if (!CreateDirectoryExW (w_oldpath->Buffer, w_newpath->Buffer, NULL)) ++ { ++ debug_printf ("CreateDirectoryExW(%S, %S, 0) failed", w_oldpath, ++ w_newpath); ++ __seterrno (); ++ return -1; ++ } ++ } ++ else if (!CopyFileExW (w_oldpath->Buffer, w_newpath->Buffer, NULL, NULL, ++ NULL, COPY_FILE_COPY_SYMLINK)) ++ { ++ __seterrno (); ++ return -1; ++ } ++ } + - /* Default technique creating a symlink. */ - buf = tp.t_get (); - cp = stpcpy (buf, SYMLINK_COOKIE); ++ return 0; ++} ++ + int + symlink_worker (const char *oldpath, path_conv &win32_newpath, bool isdevice) + { +@@ winsup/cygwin/path.cc: symlink_worker (const char *oldpath, path_conv &win32_newpath, bool isdevice) + case WSYM_nfs: + res = symlink_nfs (oldpath, win32_newpath); + __leave; ++ case WSYM_deepcopy: ++ res = symlink_deepcopy (oldpath, win32_newpath); ++ if (!res || res == -1) ++ __leave; ++ /* fall back to sysfile symlink type */ ++ wsym_type = WSYM_sysfile; ++ break; + case WSYM_native: + case WSYM_nativestrict: + res = symlink_native (oldpath, win32_newpath);@@ winsup/cygwin/spawn.cc: perhaps_suffix (const char *prog, path_conv& buf, int& e err = 0; debug_printf ("prog '%s'", prog); -- buf.check (prog, PC_SYM_FOLLOW | PC_NULLEMPTY | PC_POSIX, +- buf.check (prog, PC_SYM_FOLLOW | PC_NULLEMPTY | PC_POSIX, stat_suffixes); + buf.check (prog, PC_SYM_FOLLOW | PC_SYM_NOFOLLOW_REP | PC_NULLEMPTY | PC_POSIX, - (opt & FE_DLL) ? stat_suffixes : exe_suffixes); ++ stat_suffixes); if (buf.isdir ()) + {enable_pconvalue forMSYSMSYSenvironment variable