-
Notifications
You must be signed in to change notification settings - Fork 79
unittests now use a mock ray implementation #127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
engintoklu
wants to merge
3
commits into
master
Choose a base branch
from
fix/ray-local-mode
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+394
−17
Open
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # Copyright 2026 NNAISENSE SA | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
|
|
||
| """ | ||
| This is a mock ray implementation which does not create processes. | ||
|
|
||
| This mock implementation is meant to be used as a replacement for the | ||
| now-obsolete local mode of the actual 'ray' library. | ||
| """ | ||
|
|
||
|
|
||
| from . import util | ||
| from .core import ( | ||
| ObjectRef, | ||
| get, | ||
| init, | ||
| is_initialized, | ||
| remote, | ||
| shutdown, | ||
| ) | ||
|
|
||
| __all__ = ( | ||
| "init", | ||
| "is_initialized", | ||
| "ObjectRef", | ||
| "remote", | ||
| "get", | ||
| "util", | ||
| "shutdown", | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,257 @@ | ||
| # Copyright 2026 NNAISENSE SA | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
|
|
||
| from collections.abc import Callable, Mapping, Sequence | ||
| from typing import Any | ||
|
|
||
| try: | ||
| # We assume that 'cloudpickle' is available because it is a dependency of the 'ray' library. | ||
| # So, in this mock ray implementation, we try to use cloudpickle if possible. | ||
| import cloudpickle as pickle | ||
| except ImportError: | ||
| import pickle | ||
|
|
||
|
|
||
| def init(*args, **kwargs): | ||
| """ | ||
| Fake representation of ray initialization procedure. | ||
|
|
||
| Registers that the mock ray is initialized and ready-to-use. | ||
|
|
||
| Args: | ||
| args: Ignored. | ||
| kwargs: Ignored. | ||
| """ | ||
| init.ialized = True | ||
|
|
||
|
|
||
| init.ialized = False | ||
|
|
||
|
|
||
| def is_initialized() -> bool: | ||
| """ | ||
| Return True if 'ray' is initialized; False otherwise. | ||
| """ | ||
| return init.ialized | ||
|
|
||
|
|
||
| def shutdown(): | ||
| """ | ||
| Fake representation of ray de-initialization procedure. | ||
|
|
||
| Registers that the mock ray is de-initialized. | ||
| """ | ||
| init.ialized = False | ||
|
|
||
|
|
||
| def _pass_through_pickle(obj: Any) -> Any: | ||
| tmp = pickle.dumps(obj) | ||
| obj = pickle.loads(tmp) | ||
| return obj | ||
|
|
||
|
|
||
| def _pass_args_through_pickle(obj: Mapping | Sequence) -> Mapping | Sequence: | ||
| if isinstance(obj, Sequence): | ||
| return tuple(_pass_through_pickle(x) for x in obj) | ||
| elif isinstance(obj, Mapping): | ||
| return {k: _pass_through_pickle(v) for k, v in obj.items()} | ||
| else: | ||
| raise TypeError(f"Expected a Sequence or a Mapping, but got {type(obj)}") | ||
|
|
||
|
|
||
| class ObjectRef: | ||
| """ | ||
| Fake representation of a remote object. | ||
| """ | ||
|
|
||
| def __init__(self, obj: Any): | ||
| """ | ||
| `__init__(...)`: Initialize the `ObjectRef`. | ||
|
|
||
| Args: | ||
| obj: The object for which the remote representation will be made. | ||
| """ | ||
| self.__object = obj | ||
|
|
||
| def get(self) -> Any: | ||
| """ | ||
| Get the object stored by the so-called remote actor. | ||
|
|
||
| To mimic the transfer of the object from the remote actor to the | ||
| requester process, the object is serialized and then de-serialized, | ||
| and then returned. | ||
| """ | ||
| return _pass_through_pickle(self.__object) | ||
|
|
||
|
|
||
| class _RemoteMethod: | ||
| """ | ||
| Representation of a method of a so-called remote object. | ||
| """ | ||
|
|
||
| def __init__(self, obj: Any, attrname: str): | ||
| """ | ||
| `__init__(...)`: Initialize the `_RemoteMethod`. | ||
|
|
||
| Args: | ||
| obj: The object that owns the method to be represented. | ||
| attrname: Name of the method, as string. | ||
| """ | ||
| self.__object = obj | ||
| self.__attrname = attrname | ||
|
|
||
| def remote(self, *args, **kwargs) -> Any: | ||
| """ | ||
| Call the method with the given positional and/or keyword arguments. | ||
|
|
||
| To mimic the transfer of the arguments to the remote instance, | ||
| the arguments are serialized and then de-serialized. | ||
|
|
||
| Args: | ||
| args: Positional arguments to be passed to the method. | ||
| kwargs: Keyword arguments to be passed to the method. | ||
|
|
||
| Returns: | ||
| The result, wrapped by an `ObjectRef`. | ||
| """ | ||
| args = _pass_args_through_pickle(args) | ||
| kwargs = _pass_args_through_pickle(kwargs) | ||
| return ObjectRef(getattr(self.__object, self.__attrname)(*args, **kwargs)) | ||
|
|
||
|
|
||
| class _RemoteInstance: | ||
| """ | ||
| Representation of an object that lives on a so-called remote actor. | ||
|
|
||
| Let us say that we have an object `remote_obj` that is wrapped by | ||
| `_RemoteInstance`. A method `f` of `remote_obj` can be called like this: | ||
|
|
||
| ```python | ||
| remote_result = remote_obj.f.remote(...) | ||
| ``` | ||
|
|
||
| At this point, `remote_result` is the result of the call, wrapped by | ||
| `ObjectRef`. The actual result can be fetched by using the | ||
| function `get`. | ||
| """ | ||
|
|
||
| def __init__(self, obj: Any): | ||
| """ | ||
| `__init__(...)`: Initialize the `_RemoteInstance`. | ||
|
|
||
| Args: | ||
| obj: The object to be treated as if it lives on a remote actor. | ||
| """ | ||
| self.__object = obj | ||
|
|
||
| def __getattr__(self, attrname: str) -> _RemoteMethod: | ||
| return _RemoteMethod(self.__object, attrname) | ||
|
|
||
|
|
||
| class _RemoteCaller: | ||
| """ | ||
| Counterpart of a function or a class that is decorated via `remote`. | ||
| """ | ||
|
|
||
| def __init__(self, obj: Any): | ||
| """ | ||
| `__init__(...)`: Initialize the `_RemoteCaller`. | ||
| """ | ||
| self.__object = obj | ||
|
|
||
| def options(self, *args, **kwargs) -> "_RemoteCaller": | ||
| """ | ||
| No-op method for compatibility reasons. | ||
| """ | ||
| return self | ||
|
|
||
| def remote(self, *args, **kwargs) -> Any: | ||
| """ | ||
| Call the `remote`-decorated object with the given arguments. | ||
|
|
||
| If the decorated object is a function, the function will be called with | ||
| the given arguments, its result will be wrapped via `ObjectRef`, | ||
| and then returned. | ||
|
|
||
| If the decorated object is a class, the class will be instantiated with | ||
| the given arguments, the instance will be wrapped via `_RemoteInstance`, | ||
| and then returned. | ||
|
|
||
| Args: | ||
| args: Positional arguments to pass to the `remote`-decorated object. | ||
| kwargs: Keyword arguments to pass to the `remote`-decorated object. | ||
|
|
||
| Returns: | ||
| The result of the call, as an `ObjectRef` or as a | ||
| `_RemoteInstance`. | ||
|
|
||
| Raises: | ||
| RuntimeError: ray is not initialized | ||
| """ | ||
| if not is_initialized(): | ||
| raise RuntimeError("Cannot make a remote call without initializing 'ray' first") | ||
| args = _pass_args_through_pickle(args) | ||
| kwargs = _pass_args_through_pickle(kwargs) | ||
| result = self.__object(*args, **kwargs) | ||
| if isinstance(self.__object, type): | ||
| result = _RemoteInstance(result) | ||
| else: | ||
| result = ObjectRef(result) | ||
| return result | ||
|
|
||
|
|
||
| def remote(obj: Callable | None = None, **kwargs): | ||
| """ | ||
| Wrap a function or class so that it is handled by a so-called remote actor. | ||
|
|
||
| If the wrapped object is a function named `func`, it has to be called like | ||
| this: `func.remote(*args, **kwargs)`. | ||
|
|
||
| If the wrapped object is a class named `cls`, it has to be instantiated | ||
| like this: `cls.remote(*args, **kwargs)`. | ||
|
|
||
| To mimic the transfer of `*args` and `**kwargs` to a remote actor, | ||
| these arguments are first serialized and then de-serialized. | ||
| """ | ||
| if obj is None: | ||
| return remote | ||
| else: | ||
| return _RemoteCaller(obj) | ||
|
|
||
|
|
||
| def get(obj: ObjectRef | Sequence[ObjectRef]) -> Any: | ||
| """ | ||
| Get the value(s) of so-called remote object(s). | ||
|
|
||
| Args: | ||
| obj: An `ObjectRef`, or a sequence (e.g. list or tuple) of `ObjectRef` | ||
| instances. | ||
|
|
||
| Returns: | ||
| The value(s) of the so-called remote object(s) (as a list of values | ||
| if `obj` was given as a sequence). | ||
| """ | ||
| if isinstance(obj, ObjectRef): | ||
| return obj.get() | ||
| elif isinstance(obj, Sequence): | ||
| result = [] | ||
| for x in obj: | ||
| if isinstance(x, ObjectRef): | ||
| result.append(x.get()) | ||
| else: | ||
| raise TypeError(f"The list contains an object of this unexpected type: {type(x)}") | ||
| return result | ||
| else: | ||
| raise TypeError(f"Encountered an argument of this unexpected type: {type(obj)}") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # Copyright 2026 NNAISENSE SA | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
|
|
||
| import random | ||
| from collections.abc import Callable, Sequence | ||
| from typing import Any | ||
|
|
||
| from .core import _RemoteInstance, get | ||
|
|
||
|
|
||
| class ActorPool: | ||
| """ | ||
| A mock implementation of `ray.util.ActorPool`. | ||
| """ | ||
|
|
||
| SEED = 1 | ||
|
|
||
| def __init__(self, actors: Sequence[_RemoteInstance]): | ||
| """ | ||
| Initialize the `ActorPool`. | ||
|
|
||
| Args: | ||
| actors: List of so-called remote actors. | ||
| """ | ||
| self.__actors = actors | ||
| self.__generator = random.Random(self.SEED) | ||
|
|
||
| def map(self, fn: Callable[[_RemoteInstance, Any], Any], objects: Sequence[Any]) -> list: | ||
| """ | ||
| Pair actors with objects, and for each pair, call `fn(actor, object)`. | ||
|
|
||
| Args: | ||
| fn: The function which specifies how an object is to be used | ||
| with its associated actor. | ||
| objects: A sequence (e.g. list or tuple) of object to be processed | ||
| by the pool of actors. | ||
|
|
||
| Returns: | ||
| Processed counterparts of the objects, as a list of the same length | ||
| as `objects`. | ||
| """ | ||
| num_actors = len(self.__actors) | ||
| result = [] | ||
| for i_obj, obj in enumerate(objects): | ||
| i_actor = i_obj % num_actors | ||
| actor = self.__actors[i_actor] | ||
| result.append(fn(actor, obj)) | ||
| return get(result) | ||
|
|
||
| def map_unordered(self, fn: Callable[[_RemoteInstance, Any], Any], objects: Sequence[Any]) -> list: | ||
| """ | ||
| Like the `map` method, but the results are shuffled. | ||
|
|
||
| The reasoning behind the shuffling is to mimic the fact that | ||
| `map_unordered` of the actual 'ray' library returns the processed | ||
| results in an undeterministic order. | ||
|
|
||
| Also note that, for reproducibility during tests, the shuffling | ||
| depends on a constant random seed, which is stored by class-level | ||
| constant `SEED`. | ||
| """ | ||
| result = self.map(fn, objects) | ||
| self.__generator.shuffle(result) | ||
| return result | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: nnaisense/evotorch
Length of output: 167
🌐 Web query:
Ray ActorPool.map returns generator iterator return type map_unordered unordered iterator yields results as they complete💡 Result:
In Ray, both ray.util.ActorPool.map and ray.util.ActorPool.map_unordered return a Python generator iterator [1][2][3]. The key distinction between the two methods lies in the ordering of the returned results [1][2]: - ray.util.ActorPool.map returns an ordered iterator that yields results corresponding to the input list order, as they finish [1]. - ray.util.ActorPool.map_unordered returns an unordered iterator that yields results as they complete [2][4]. Because map_unordered does not enforce any ordering constraints, it can be more efficient than map in scenarios where tasks have variable execution times, as it allows results to be processed as soon as they become available rather than waiting for preceding tasks in the input sequence to complete [2][4]. For both methods, it is important to note that you must iterate over the returned iterator to actually trigger and consume the computation [1][5]. If you simply call the method without iterating, the tasks may not be fully executed or the results may not be retrieved as expected [1].
Citations:
🌐 Web query:
rAY util ActorPool map_unordered documentation returns generator iterator list💡 Result:
Ray’s
ray.util.ActorPool.map_unordered(...)returns an iterator (a generator object) that yields results “as they finish”, i.e., unordered with respect to the input order. [1] Concretely, the implementation defines an innerget_generator()that doeswhile self.has_next(): yield self.get_next_unordered()and then returnsget_generator(); so you can iterate it directly or consume it (e.g.,list(pool.map_unordered(...))). [2] The docs formap_unordereddescribe this as “returning an unordered iterator” whose yielded results are produced as tasks complete, which may be more efficient than orderedmap()when some computations take longer. [1]Citations:
🏁 Script executed:
Repository: nnaisense/evotorch
Length of output: 167
Preserve Ray iterator semantics in the
ActorPoolmock.Ray’s
ray.util.ActorPool.map/map_unorderedreturn iterator/generator objects (lazy; results are produced as the iterator is consumed, withmap_unorderedyielding as tasks complete). This mock eagerly materializes a full list inmap, andmap_unorderedshuffles only after fully realizing the list, so partial iteration /next(...)-style consumption won’t match Ray behavior. https://docs.ray.io/en/latest/ray-core/api/doc/ray.util.ActorPool.map.html https://docs.ray.io/en/latest/ray-core/api/doc/ray.util.ActorPool.map_unordered.html🤖 Prompt for AI Agents