Add Alteryx Engine methods to 'logger' or generic 'alteryx_engine' namespace

[daqieq]

This is generic version of this other issue -- #6.

While working on a tool using SnakePlane recently, I needed a temp directory for files. While I ended up using the Python tempfile package instead, I did try finding the TempFilePath workflow constant at one point.

It appears that the alteryx_engine inside AyxPlugin is hidden from the 'pilot', and as such, the output_message method is only exposed to the 'pilot' through the logger. So to get access to get_constant method, I added it temporarily to the self.logging SimpleNamespace following the pattern, and was able to use the following in Pdb:

(Pdb) logger.get_constant(0)
('Engine.GuiInteraction', '', '0', False)
(Pdb) logger.get_constant(1)
('Engine.IterationNumber', '', '0', True)
(Pdb) logger.get_constant(2)
('Engine.ModuleDirectory', '', 'C:\\Users\\<user>\\Documents\\~Alteryx\\snakeplane-master\\pilot\\debug_workflows\\', False)
(Pdb) logger.get_constant(3)
('Engine.TempFilePath', '', 'C:\\Users\\<user>\\AppData\\Local\\Temp\\Engine_7016_d592e2b9ab1c4a87b2c08afb64a01e81_\\', False)
(Pdb) logger.get_constant(4)
('Engine.Version', '', '2019.2.5.62427', False)
(Pdb) logger.get_constant(5)
('Engine.WorkflowDirectory', '', 'C:\\Users\\<user>\\Documents\\~Alteryx\\snakeplane-master\\pilot\\debug_workflows\\', False)
(Pdb) logger.get_constant(6)
('Engine.WorkflowFileName', '', './debug_workflows/SFTPInput.yxmd', False)

Does alteryx_engine need to be hidden in this fashion? If so, can we add some more features of the engine to either the logger, or a new SimpleNamespace? When I check dir(sdk.AlteryxEngine) I see the following methods that could be valuable to 'pilot':

[..., 'create_connect_metadata', 'create_temp_file_name', 'debug_message_out', 'decrypt_password', 'field_conversion_error_limit', 'get_constant', 'get_init_var', 'output_message', 'output_tool_progress', 'pre_sort', 'xmsg']

Several of these look indispensable! And the change to helper_classes for get_constant was simple enough:

            ...
            get_constant=partial(
                self._engine_vars.alteryx_engine.get_constant,
                self._engine_vars.n_tool_id,
            ),
            ...

Thank you for considering!

[dme722]

We will definitely take all of this into consideration! I think that for your uses, you can use the alteryx_engine directly for the time being, and we will eventually make them more accessible directly through snakeplane.

[ashtonwalden]

@daqieq: I ran into a similar issue with my projects. Here is a code snippet that creates a temp folder that is managed by Alteryx Engine using the "create_temp_file_name" that you pointed out. Using the constants would be ideal, but this is a good workaround for the time being.

file_path = output_mgr._plugin._engine_vars.alteryx_engine.create_temp_file_name( extension="txt", options=0 )
project_path = file_path[: file_path.rfind("\\")] + "\\temp"

Note: When running in designer, a lock file is placed in the root temp folder. That's why I am creating a sub-directory.

[daqieq]

@AshtonUPS - Were you able to access output_mgr._plugin.engine_vars.alteryx_engine from the 'pilot'? I.e. in pilot\src\ToolName\engine\src\main.py
In my testing, I had to go into helper_classes.py and add stuff there to get it to work. I can try again this week and see if I was just doing something wrong.

@dme722 - Do you mean that I can use alteryx_engine in main.py? If so, I think I'm missing how to do that. When I'm in pdb (using set_trace()) I don't 'see' the alteryx_engine object. I haven't opened my project in a few days because I've been busy on some other things, but I'm hoping to get back to it soon and I can try it again.

[johnkabler]

@daqieq Correct, you SHOULD be able to access the alteryx_engine via output_mgr in either your process_data or build_metadata decorated functions you've defined. If you look in the helper_classes.py file in snakeplane, at line 399, you'll see how the "get_temp_file_path" method on OutputManager just calls the private _plugin object to access the alteryx_engine object.

We are going to see if we can add some syntactic sugar for some of these more useful methods in the future.