Suggestions for development in an OSX environment?

[mistermocha]

I'm sure this is a bit of a stretch to ask, but here goes anyway...

We're looking to implement pylibacl, but we do most of our development in an OSX environment to push out to CentOS hosts in production. Far as I can tell, there's no libacl available for OSX. While I'm aware of doing development in docker and such, do you have any suggestions for straight OSX development and testing?

[iustin]

Hey,

OSX actually should have acls, although not provided by a library called libacl. At least this page suggests so (despite the URL, it claims to be a Mac OS X man page).

Also, way back, somebody reported pylibacl works on Mac OS X, so I commited 06dd591. Now, since there's no CI that tests Mac OS (and in general, because I don't have access to Mac OS), this might have bit rotted away, but would be good to resurrect.

What fails if you simply try to build on Mac OS?

[mistermocha]

Thanks for the prompt follow-up!

The gcc build undnerneath can't find acl.h in a py3.7 build

 acl.c:27:21: fatal error: sys/acl.h: No such file or directory
  #include <sys/acl.h>
                      ^
 compilation terminated.
 error: command 'gcc' failed with exit status 1

We're using pants for our build

[iustin]

The man-page that I quoted suggests that the acl functions are provided by libc, so I wonder - can you comment out that include, and see what the result is?

[mistermocha]

Hi there, just dropping in with some more details, haven't been able to dig into this until this morning... and still no success.

For more context, I'm on Catalina 10.5.7 using Python 3.7. The issues at hand seem primarily around the C libraries. The commit you referred above is specifically in setup.py and the Makefile never gets that far in make test before throwing an exception.

Here's my make test output, showing that there are missing symbols (likely from missing sys/acl.h, but I'm not sure, will keep hammering)

Testing with python3.7
running build_ext
building 'posix1e' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c acl.c -o build/temp.macosx-10.9-x86_64-3.7/acl.o
acl.c:77:37: error: use of undeclared identifier 'ACL_READ'
static acl_perm_t holder_ACL_READ = ACL_READ;
                                    ^
acl.c:78:38: error: use of undeclared identifier 'ACL_WRITE'
static acl_perm_t holder_ACL_WRITE = ACL_WRITE;
                                     ^
2 errors generated.
error: command 'gcc' failed with exit status 1
make: *** [test] Error 1

Additionally, the acl.h reference you linked is 20 years old and I cannot locate it in any of my current xcode-provided libraries. I've also looked about in macports & homebrew for any such equivalents with no such luck.

[mistermocha]

Well the good news is I found sys/acl.h, now I'm looking at how to do the correct linking into Makefile or setup.py. Suggestions welcome

$ find /Library/Developer/CommandLineTools -name 'acl.h' -print
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk 1/usr/include/sys/acl.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/acl.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/acl.h```

[iustin]

Hmm. sys/acl.h is already included, see acl.c, line 27. However, if that doesn't define ACL_READ… I really need to find a way to test myself on Mac OS. Right now I don't have any other suggestions :(

[mistermocha]

The fields are just slightly different in OSX:

typedef enum {
  ACL_READ_DATA   = __DARWIN_ACL_READ_DATA,
  ACL_LIST_DIRECTORY  = __DARWIN_ACL_LIST_DIRECTORY,
  ACL_WRITE_DATA    = __DARWIN_ACL_WRITE_DATA,
  ACL_ADD_FILE    = __DARWIN_ACL_ADD_FILE,
  ACL_EXECUTE   = __DARWIN_ACL_EXECUTE,
  ACL_SEARCH    = __DARWIN_ACL_SEARCH,
  ACL_DELETE    = __DARWIN_ACL_DELETE,
  ACL_APPEND_DATA   = __DARWIN_ACL_APPEND_DATA,
  ACL_ADD_SUBDIRECTORY  = __DARWIN_ACL_ADD_SUBDIRECTORY,
  ACL_DELETE_CHILD  = __DARWIN_ACL_DELETE_CHILD,
  ACL_READ_ATTRIBUTES = __DARWIN_ACL_READ_ATTRIBUTES,
  ACL_WRITE_ATTRIBUTES  = __DARWIN_ACL_WRITE_ATTRIBUTES,
  ACL_READ_EXTATTRIBUTES  = __DARWIN_ACL_READ_EXTATTRIBUTES,
  ACL_WRITE_EXTATTRIBUTES = __DARWIN_ACL_WRITE_EXTATTRIBUTES,
  ACL_READ_SECURITY = __DARWIN_ACL_READ_SECURITY,
  ACL_WRITE_SECURITY  = __DARWIN_ACL_WRITE_SECURITY,
  ACL_CHANGE_OWNER  = __DARWIN_ACL_CHANGE_OWNER,
  ACL_SYNCHRONIZE   = __DARWIN_ACL_SYNCHRONIZE,
} acl_perm_t;

I've tried doing overrides in acl.c ...

#ifdef __APPLE__
static acl_perm_t holder_ACL_EXECUTE = ACL_EXECUTE;
static acl_perm_t holder_ACL_READ = ACL_READ_DATA;
static acl_perm_t holder_ACL_WRITE = ACL_WRITE_DATA;
#else
static acl_perm_t holder_ACL_EXECUTE = ACL_EXECUTE;
static acl_perm_t holder_ACL_READ = ACL_READ;
static acl_perm_t holder_ACL_WRITE = ACL_WRITE;
#endif

... but the tests still bomb out ...

tests/test_acls.py:47: in <module>
    (ACL_READ, "read", Permset.read),
E   NameError: name 'ACL_READ' is not defined

Dropping pdb.set_trace() in the test to examine the output of posix1e for appropriate pyojbects showed the effective calls were missing entirely

(Pdb) dir(posix1e)
['ACL', 'ACL_TYPE_ACCESS', 'ACL_TYPE_DEFAULT', 'HAS_ACL_CHECK', 'HAS_ACL_ENTRY', 'HAS_ACL_FROM_MODE', 'HAS_COPY_EXT', 'HAS_EQUIV_MODE', 'HAS_EXTENDED_CHECK', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'delete_default']

I'm not much of a C developer so I'm trying my best here to fit this in properly... I'm happy to commit the right elements to make this build if I can get this figured out.

[PeqNP]

Same issue, @mistermocha. Were you able to resolve this?

[PeqNP]

Going to brain dump what I've learned so far in the hopes that someone can carry the torch.

As @mistermocha stated, tests fail and the posix1e has no definitions for ALC_READ, etc.

You can either #define ACL_READ to map to ACL_READ_DATA or override like mocha did. Compilation will succeed, but the tests will fail to run.

The setup.py doesn't declare Darwin to have level 2 (HAVE_LEVEL2) support. That's probably true. The result is that none of the code that uses ACL_READ appears to be compiled as it is all related to having level 2 support. I'm guessing that's why you get the build warnings saying that ACL_READ, ACL_WRITE, et al. are not used.

Regarding level 2 support; I tried looking through /Library and /usr. Except for finding acl.h in /Library, I couldn't find a header that contained ACL_USER, ACL_GROUP, et al., which appears to be required for level 2 support. My gut tells me that level 2 is supported but it's implemented in a way that is not compatible with the Linux libacl.

I'm going to see if there's a way to at least get ACL_READ and friends to bind to posix1e.

[PeqNP]

Well, I figured out how to bind ACL_READ and friends.

You can do this by pulling this out of the #ifdef HAVE_LEVEL2 block:

PyModule_AddIntConstant(m, "ACL_READ", ACL_READ);
...

This raised an issue where Permset is not defined when running the tests. I tried to add a new macro HAVE_DARWIN in an attempt to bind Permset while avoiding libacl types (ACL_GROUP, etc.) but quickly realized you can't have one without the other. The Entry type requires user, group, etc.

Damn.

[PeqNP]

It seems like the tests are testing level 2 features. I think that's the main issue. They're not going to run on macOS.

I tried using the lib using the example here (https://pylibacl.k1024.org/module.html) doesn't appear to work:

#!/usr/bin/env python3
import posix1e
acl = posix1e.ACL(file="file.txt")
print(acl)

provides the following:

Traceback (most recent call last):
  File "/path/to/test_acl.py", line 4, in <module>
    acl = posix1e.ACL(file="file.txt")
OSError: [Errno 22] Invalid argument: 'file.txt'

Yes, there is a file.txt in the same directory where the test_acl.py script is running. I've also tried giving it the full path to the file.

That's about as far as I'm going to go :)