A helper interface for requesting mesh modifications from the device. This is intended to be used in a three-stage command pattern: More...

#include <NgpModRequests.hpp>

Inheritance diagram for mundy::mesh::NgpModRequestsT< NgpMemSpace >:

[legend]

Public Types
using	memory_space = NgpMemSpace
using	ticket_id = size_t
using	control_space = Kokkos::SharedSpace
using	host_state_space = Kokkos::HostSpace

Public Member Functions
Constructors / Destructors
	NgpModRequestsT ()
	Default constructor.
	NgpModRequestsT (const NgpModRequestsT &)=default
NgpModRequestsT &	operator= (const NgpModRequestsT &)=default
	NgpModRequestsT (NgpModRequestsT &&)=default
NgpModRequestsT &	operator= (NgpModRequestsT &&)=default
	~NgpModRequestsT ()=default
Accessors for request classes
NgpRequestEntitiesNewIdsT< NgpMemSpace >	request_entities_new_ids (const stk::mesh::PartVector &parts) const
	Get the entity request class for the given partition key (for requests of entities with new Ids).
NgpRequestEntitiesNewIdsT< NgpMemSpace >	request_entities_new_ids (const stk::mesh::Part &part) const
NgpRequestEntitiesNewIdsT< NgpMemSpace >	request_entities_new_ids (const ClassVector &classes) const
	Class-based overloads: entity membership is determined per rank by `impl::populate_entity_rank_parts`, which handles primary classes, sets, and induced sets correctly.
NgpRequestEntitiesNewIdsT< NgpMemSpace >	request_entities_new_ids (const Class &cls) const
NgpRequestEntitiesKnownIdsT< NgpMemSpace >	request_entities_known_ids (const stk::mesh::PartVector &parts) const
	Get the entity request class for the given partition key (for requests of entities with known Ids). Aka you assign the Id at request time instead of it being automatically generated by STK.
NgpRequestEntitiesKnownIdsT< NgpMemSpace >	request_entities_known_ids (const stk::mesh::Part &part) const
NgpRequestEntitiesKnownIdsT< NgpMemSpace >	request_entities_known_ids (const ClassVector &classes) const
	Class-based overloads: entity membership is determined per rank by `impl::populate_entity_rank_parts`.
NgpRequestEntitiesKnownIdsT< NgpMemSpace >	request_entities_known_ids (const Class &cls) const
NgpDestroyEntitiesT< NgpMemSpace >	destroy_entities () const
	Get the destroy entity request class.
NgpRequestConnectionsT< NgpMemSpace >	request_connections () const
	Get the connection request class.
NgpDestroyConnectionsT< NgpMemSpace >	destroy_connections () const
	Get the destroy connection request class.
NgpRequestLinkRelationsT< NgpMemSpace >	request_links (LinkData &link_data) const
	Get or create the link-relation request helper for the given LinkData.
Control plane (HOST only)
void	activate_host ()
	Sets the active memory space to host for all request helpers and ticket issuers.
void	activate_device ()
	Sets the active memory space to device for all request helpers and ticket issuers.

Mod cycle management
void	reset ()
	Clears all internal request data to prepare for a fresh modification cycle.
void	finalize_counts ()
	Finalize counts for all request classes. Users may no longer claim tickets after this call.
void	process_requests (stk::mesh::BulkData &bulk_data)
	Process all requests in all request classes. The order goes: 1) Entity requests 2) Connection requests 3) Destroy connection requests 4) Destroy entity requests.

Detailed Description

template<typename NgpMemSpace>
class mundy::mesh::NgpModRequestsT< NgpMemSpace >

Manages all modification requests in a given NGP memory space.

Stage 1. Claim tickets

Just like a ticketing system at a deli counter, you first claim tickets from a ticketing machine. You haven't actually made any purchases yet but you have stated your intent to make a certain number of orders (requests).

Stage 2. Make requests

After all of your claims are in and you've finalized the counts, you can then use your claimed tickets to make requests.

Stage 3. Fetch

After all requests are processed, use the same ticket to fetch the result of your request (if any).

During each stage, there are multiple types of requests you can make (requesting entities, requesting connections, destroying entities, etc). Each type of request has its own interface and its own ticket issuer. For example, if you want to request 10 new entities and 10 new connections, you would claim 10 tickets from the entity ticket issuer and 10 tickets from the connection ticket issuer. You would then use the entity tickets to make your entity requests and the connection tickets to make your connection requests (potentially requesting connections to requested entities), allowing you to mix and match requests as needed.

Note: For the time being, you must use each ticket you claim (aka, you cannot over-claim).; You can claim tickets and make requests from either the host or device, but not both simultaneously.

Example usage 1: Create N spheres (N elems + N nodes) and connect each sphere to its node

size_t num_spheres = ...;
stk::mesh::BulkData& bulk_data = ...;
stk::mesh::Part& sphere_part = ...;
 
NgpModRequests reqs;
auto req_spheres = reqs.request_entities_new_ids(sphere_part);
auto req_conns = reqs.request_connections();
 
// Stage 1: Claim tickets
reqs.activate_host();  // Ensures that claim can be called from host
req_spheres.element_tickets().claim(num_spheres);
req_spheres.node_tickets().claim(num_spheres);
req_conns.tickets().claim(num_spheres);
reqs.finalize_counts();
 
// Stage 2: Request
reqs.activate_device();  // Ensures that request can be made from device
Kokkos::parallel_for(
    "RequestSpheres", Kokkos::RangePolicy<size_t>(0, num_spheres), KOKKOS_LAMBDA(size_t ticket) {
      FutureEntity future_sphere = req_spheres.request_element(ticket);
      FutureEntity future_node = req_spheres.request_node(ticket);
      req_conns.request(ticket, future_sphere, future_node, 0);
    });
reqs.process_requests(bulk_data);
 
// Stage 3: Fetch
Kokkos::parallel_for(
    "UseSpheres", Kokkos::RangePolicy<size_t>(0, num_spheres), KOKKOS_LAMBDA(size_t ticket) {
      stk::mesh::Entity sphere = req_spheres.get_entity(ticket, stk::topology::ELEMENT_RANK);
      stk::mesh::Entity node = req_spheres.get_entity(ticket, stk::topology::NODE_RANK);
 
      // Do something with sphere and its node...
    });

Example usage 2: Divide all bacteria that exceeds twice the starting length into two bacteria Bacteria are spherocylinder segments (PARTICLE topology elements with a single node at their center) They have a "length", "radius", and "orientation" field Their center nodes have a "coords" field

Because division is delayed, we also need bacteria to store the ticket for fetching their new child later

We will assume that these are already declared and that you have wrapped them into a single bacteria aggregate

auto ngp_bacteria_data = get_updated_ngp_aggregate(bacteria_data);
auto ngp_mesh = /* refreshed NgpMesh view for your workflow */;
 
size_t num_bacteria = ...;  // total number of bacteria
double starting_length = ...;
stk::mesh::BulkData& bulk_data = ...;
stk::mesh::Part& bacteria_part = ...;
 
NgpModRequests reqs;
auto req_bacteria = reqs.request_entities_new_ids(bacteria_part);
auto req_conns = reqs.request_connections();
 
// Stage 1: Claim tickets
reqs.activate_device();
stk::mesh::for_each_entity_run(
    ngp_mesh, stk::topology::ELEM_RANK, ngp_bacteria_data.get<SELECTOR>(),
    KOKKOS_LAMBDA(stk::mesh::FastMeshIndex bacteria_index) {
      double centerline_length = ngp_bacteria_data.get<LENGTH>(bacteria_index);
      double radius = ngp_bacteria_data.get<RADIUS>(bacteria_index);
      double total_length = centerline_length + 2.0 * radius;
      if (total_length > 2.0 * starting_length) {
        size_t ticket = req_bacteria.element_tickets().claim();
        req_bacteria.node_tickets().claim();
        req_conns.tickets().claim();
 
        // Store the ticket for later use
        ngp_bacteria_data.get<CHILD_TICKET>(bacteria_index) = ticket;
      }
    });
reqs.finalize_counts();
 
// Stage 2: Request
reqs.activate_device();
stk::mesh::for_each_entity_run(
    ngp_mesh, stk::topology::ELEM_RANK, ngp_bacteria_data.get<SELECTOR>(),
    KOKKOS_LAMBDA(stk::mesh::FastMeshIndex bacteria_index) {
      double centerline_length = ngp_bacteria_data.get<LENGTH>(bacteria_index);
      double radius = ngp_bacteria_data.get<RADIUS>(bacteria_index);
      double total_length = centerline_length + 2.0 * radius;
      if (total_length > 2.0 * starting_length) {
        // One new bacteria since the parent becomes one of the children
        size_t ticket = ngp_bacteria_data.get<CHILD_TICKET>(bacteria_index);
        FutureEntity future_bacteria = req_bacteria.request_element(ticket);
        FutureEntity future_node = req_bacteria.request_node(ticket);
        req_conns.request(ticket, future_bacteria, future_node, 0);
      }
    });
reqs.process_requests(bulk_data);
 
// Stage 3: Fetch
stk::mesh::for_each_entity_run(
    ngp_mesh, stk::topology::ELEM_RANK, ngp_bacteria_data.get<SELECTOR>(),
    KOKKOS_LAMBDA(stk::mesh::FastMeshIndex bacteria_index) {
      double centerline_length = ngp_bacteria_data.get<LENGTH>(bacteria_index);
      double radius = ngp_bacteria_data.get<RADIUS>(bacteria_index);
      double total_length = centerline_length + 2.0 * radius;
      if (total_length > 2.0 * starting_length) {
        size_t ticket = ngp_bacteria_data.get<CHILD_TICKET>(bacteria_index);
 
        stk::mesh::Entity new_bacteria = req_bacteria.get_entity(ticket, stk::topology::ELEMENT_RANK);
        stk::mesh::Entity center_node = req_bacteria.get_entity(ticket, stk::topology::NODE_RANK);
 
        // The parent becomes the leftmost child and the new bacteria is the rightmost child
        //
        // New length = length / 2 - radius
        // New centers = old center +/- (tangent * (new length / 2 + radius))
        // ...
      }
    });

The above design is amicable to a future python interface:

# Example usage 1 (but in python): Create N spheres (N elems + N nodes)
num_spheres = ...
our_proc_id = bulk_data.parallel_rank()
 
reqs = NgpModRequests()
req_spheres = reqs.request_entities(sphere_part)
req_nodes = reqs.request_entities()
req_conns = reqs.request_connections()
 
# Stage 1: Claim tickets
with reqs.host():
  req_spheres.tickets().claim(num_spheres)
  req_nodes.tickets().claim(num_spheres)
  req_conns.tickets().claim(num_spheres)
 
reqs.finalize_counts()
 
# Stage 2: Request
with reqs.device():
  plan = reqs.plan()  # collects device ops that must be executed
 
  t = req_spheres.tickets().range(num_spheres)  # 0..num_spheres-1, typed object
  future_spheres = req_spheres.request(t, our_proc_id)
  future_nodes = req_nodes.request(t, our_proc_id)
  req_conns.request(t, future_spheres, future_nodes)
  ticket_expr1.exec()
 
reqs.process_requests();
 
# Stage 3: Fetch
with reqs.device():
  ticket_expr2 = make_ticket_expr(0, num_spheres)
  spheres = req_spheres.get_entity(ticket_expr2)
  nodes = req_nodes.get_entity(ticket_expr2)
  center(nodes) = ...
  radius(spheres) = ...
  ticket_expr2.exec()