{
 "cells": [
  {
   "cell_type": "raw",
   "id": "eeaf58f2",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    ".. _running-programs:\n",
    "\n",
    "Running Cicada Programs\n",
    "=======================\n",
    "\n",
    "Because a Cicada \"program\" is actually multiple copies of a single program running simultaneously (typically on multiple machines), bootstrapping the entire process can be tricky.  It would quickly become annoying if you had to execute five separate command lines every time you wanted to run a program with five players, especially while coding and debugging!\n",
    "\n",
    "To streamline the process, Cicada provides tools to assist with the following use-cases:"
   ]
  },
  {
   "cell_type": "raw",
   "id": "1b94d3be",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Learning and Development\n",
    "------------------------\n",
    "\n",
    "Whether you're new to Cicada or developing new algorithms and programs, we assume for this use-case that it's acceptable to have all of your players running on the same machine; although this greatly reduces privacy, it dramatically reduces the complexity and overhead of running a Cicada program.\n",
    "\n",
    "During this phase, we recommend that you create a top-level function that wraps your program, then use :any:`SocketCommunicator.run` to execute that function using multiple players running on the local machine:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0e3f33ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "# my-program.py\n",
    "from cicada.communicator import SocketCommunicator\n",
    "\n",
    "def main(communicator):\n",
    "    # Your program here\n",
    "    pass\n",
    "    \n",
    "SocketCommunicator.run(world_size=5, fn=main);"
   ]
  },
  {
   "cell_type": "raw",
   "id": "a834ebd0",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "With this approach, your program can easily be run in a Jupyter notebook (as we do throughout this documentation), or executed from the command-line with a single command:\n",
    "\n",
    ".. code-block:: bash\n",
    "\n",
    "    $ python my-program.py\n",
    "    ... Your program output here\n",
    "    \n",
    "    \n",
    "In addition to running your function in parallel, :any:`SocketCommunicator.run` will gather its return values, returning them as a list in rank order.  For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ebb2ae82",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Player 0 result: 'Hello from player 0!'\n",
      "Player 1 result: 'Hello from player 1!'\n",
      "Player 2 result: 'Hello from player 2!'\n",
      "Player 3 result: 'Hello from player 3!'\n",
      "Player 4 result: 'Hello from player 4!'\n"
     ]
    }
   ],
   "source": [
    "def main(communicator):\n",
    "    return f\"Hello from player {communicator.rank}!\"\n",
    "\n",
    "results = SocketCommunicator.run(world_size=5, fn=main)\n",
    "for rank, result in enumerate(results):\n",
    "    print(f\"Player {rank} result: {result!r}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a2060d7",
   "metadata": {},
   "source": [
    "Special return values are used to indicate players that fail, whether by raising an exception or dying unexpectedly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ee4888fc",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Comm world player 3 failed: RuntimeError('Ahhhh! YOU GOT ME!')\n",
      "Comm world player 4 failed: Terminated(exitcode=-9)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Player 0 result: 'Hello from player 0!'\n",
      "Player 1 result: 'Hello from player 1!'\n",
      "Player 2 result: 'Hello from player 2!'\n",
      "Player 3 result: Failed(exception=RuntimeError('Ahhhh! YOU GOT ME!'))\n",
      "Player 4 result: Terminated(exitcode=-9)\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import signal\n",
    "\n",
    "def main(communicator):\n",
    "    # Examples of normal return values.\n",
    "    if communicator.rank in [0, 1, 2]:\n",
    "        return f\"Hello from player {communicator.rank}!\"\n",
    "    # Example of a failure that raises an exception.\n",
    "    if communicator.rank == 3:\n",
    "        raise RuntimeError(\"Ahhhh! YOU GOT ME!\")\n",
    "    # Example of a process that dies unexpectedly.\n",
    "    if communicator.rank == 4:\n",
    "        os.kill(os.getpid(), signal.SIGKILL)\n",
    "\n",
    "results = SocketCommunicator.run(world_size=5, fn=main)\n",
    "for rank, result in enumerate(results):\n",
    "    print(f\"Player {rank} result: {result!r}\")"
   ]
  },
  {
   "cell_type": "raw",
   "id": "f5717deb",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "As you can see, the :class:`~cicada.communicator.socket.Terminated` class indicates players whose process exited unexpectedly, and stores the process exit code, while :class:`~cicada.communicator.socket.Failed` indicates players that raised an exception, and includes a copy of the exception that you can use to better understand what went wrong.  Iit also includes a traceback object, which can be used to identify which line in the code raised the exception:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "4b143f8e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Comm world player 3 failed: RuntimeError('Ahhhh! YOU GOT ME!')\n",
      "Comm world player 4 failed: Terminated(exitcode=-9)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Player 0 result: 'Hello from player 0!'\n",
      "Player 1 result: 'Hello from player 1!'\n",
      "Player 2 result: 'Hello from player 2!'\n",
      "Player 3 result: Failed(exception=RuntimeError('Ahhhh! YOU GOT ME!'))\n",
      "Player 3: Traceback (most recent call last):\n",
      "  File \"/Users/tshead/src/cicada-mpc/cicada/communicator/socket/__init__.py\", line 786, in launch\n",
      "    result = fn(communicator, *args, **kwargs)\n",
      "             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
      "  File \"/var/folders/tl/h2xygzzn1154jzjn_n01x860001l4n/T/ipykernel_7011/308613329.py\", line 9, in main\n",
      "    raise RuntimeError(\"Ahhhh! YOU GOT ME!\")\n",
      "RuntimeError: Ahhhh! YOU GOT ME!\n",
      "\n",
      "Player 4 result: Terminated(exitcode=-9)\n"
     ]
    }
   ],
   "source": [
    "from cicada.communicator.socket import Failed\n",
    "\n",
    "def main(communicator):\n",
    "    # Examples of normal return values.\n",
    "    if communicator.rank in [0, 1, 2]:\n",
    "        return f\"Hello from player {communicator.rank}!\"\n",
    "    # Example of a failure that raises an exception.\n",
    "    if communicator.rank == 3:\n",
    "        raise RuntimeError(\"Ahhhh! YOU GOT ME!\")\n",
    "    # Example of a process that dies unexpectedly.\n",
    "    if communicator.rank == 4:\n",
    "        os.kill(os.getpid(), signal.SIGKILL)\n",
    "\n",
    "results = SocketCommunicator.run(world_size=5, fn=main)\n",
    "for rank, result in enumerate(results):\n",
    "    print(f\"Player {rank} result: {result!r}\")\n",
    "    if isinstance(result, Failed):\n",
    "        print(f\"Player {rank}: {result.traceback}\")"
   ]
  },
  {
   "cell_type": "raw",
   "id": "2ac6075a",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Preparing for Deployment\n",
    "------------------------\n",
    "\n",
    "Once you've worked-out the bugs in your new program, it's time to start thinking about deployment.  To truly protect the privacy of your program's players, they will each need to execute your code on a separate machine that they control. This means that you'll have to replace :any:`SocketCommunicator.run` with an alternative.  A good option is the :any:`SocketCommunicator.connect` function, which works with the :ref:`cicada` command to create a communicator that you can pass to your main function yourself::\n",
    "\n",
    "    # my-program.py\n",
    "    from cicada.communicator import SocketCommunicator\n",
    "\n",
    "    def main(communicator):\n",
    "        # Your program here\n",
    "        pass\n",
    "\n",
    "    with SocketCommunicator.connect() as communicator:\n",
    "        main(communicator)\n",
    "\n",
    "Now, when you run your program with the :ref:`cicada` command, it will setup environment variables that :any:`SocketCommunicator.connect` reads at runtime.  To get started, try using `cicada run` to execute your program on the local machine:\n",
    "\n",
    ".. code-block:: bash\n",
    "\n",
    "    $ cicada run --world-size 5 my-program.py\n",
    "    ... Your program output here\n",
    "    \n",
    "Although this is still running all processes locally, it's going through all of the steps of setting up the environment and running the separate processes, so it's a good sanity check that nothing broke when you changed your startup code.\n",
    "\n",
    ".. tip::\n",
    "\n",
    "    Advanced users may wish to bypass the :ref:`cicada` command by setting the CICADA_WORLD_SIZE, CICADA_RANK, CICADA_ADDRESS, and CICADA_ROOT_ADDRESS environment variables themselves.  This can be handy if you're running the same Cicada program repeatedly on hardware that doesn't change."
   ]
  },
  {
   "cell_type": "raw",
   "id": "1e93dab2",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Interactive Programs\n",
    "--------------------\n",
    "\n",
    "In addition to simulating what it's like to start a full-fledged program running on multiple machines, :ref:`cicada` can help if you're running an interactive program, i.e. any program that will prompt users for command-line input.  Because `stdin` can't be shared between multiple processes, the only way for each process to get user input is to execute them in separate shells.  If you're working on a system with `X11` installed, :ref:`cicada` can run each process in a separate xterm:\n",
    "\n",
    ".. code-block:: bash\n",
    "\n",
    "    $ cicada run --world-size 5 --frontend xterm my-program.py\n",
    "    \n",
    "... when you do this, :ref:`cicada` creates as many new xterms as players, so that each can prompt for input and generate output without stepping on the others.  In fact, you may find that you prefer working this way even if your program *isn't* interactive, since the outputs from each player are neatly separated into their own windows.\n",
    "\n",
    "One thing to keep in mind is that when your program ends, the xterms will immediately close, which may be a problem if you need to check your program's output. To prevent this from happening, run your program with the `--inspect` flag, which will drop into an interactive Python interpreter when your program ends, allowing you to view the program output at your leisure, and even inspect the program's internal state:\n",
    "\n",
    ".. code-block:: bash\n",
    "\n",
    "    $ cicada run --world-size 5 --frontend xterm --inspect my-program.py\n",
    "\n",
    "If you don't have `X11` support, or you don't like the clutter of multiple xterms, :ref:`cicada` can use `tmux` to run each player in a separate tmux window pane:\n",
    "\n",
    ".. code-block:: bash\n",
    "\n",
    "    $ cicada run --world-size 5 --frontend tmux --inspect my-program.py"
   ]
  },
  {
   "cell_type": "raw",
   "id": "e178c111",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Deployment\n",
    "----------\n",
    "\n",
    "OK! You've run all the tests, and you're finally ready to do this thing for real.  Now you are going to have to run each player separately, on separate machines. To do this, you'll use the `cicada start` command instead of `cicada run`.  Before you begin, you'll need to keep two things in mind:\n",
    "\n",
    "* Every player must have a public IP address that is visible to the other players.\n",
    "* Every player must know the public IP address and port number that the root (rank 0) player is using.\n",
    "\n",
    "Assuming the above is true, you can use `cicada start` command to start each player individually. For example, assuming that 192.168.0.4 is a public IP address of player 0:\n",
    "\n",
    ".. important::\n",
    "\n",
    "   Run each of the following commands in a shell on a different machine!\n",
    "\n",
    ".. code-block:: bash\n",
    "\n",
    "    alice@host1 $ cicada start --world-size 5 --rank 0 --address tcp://192.168.0.4:25252 my-program.py\n",
    "      bob@host2 $ cicada start --world-size 5 --rank 1 --root-address tcp://192.168.0.4:25252 my-program.py\n",
    "    carol@host3 $ cicada start --world-size 5 --rank 2 --root-address tcp://192.168.0.4:25252 my-program.py\n",
    "      dan@host4 $ cicada start --world-size 5 --rank 3 --root-address tcp://192.168.0.4:25252 my-program.py\n",
    "     erin@host5 $ cicada start --world-size 5 --rank 4 --root-address tcp://192.168.0.4:25252 my-program.py\n",
    "\n",
    "Here, player 0 uses `--address` to explicitly specify a public address and port number, and the remaining players specify the same information with `--root-address`.\n",
    "\n",
    ".. tip::\n",
    "\n",
    "    For practice, you can use `cicada start` to run processes on the local machine too."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}