What is a good use for the second argument of the PYU constructor?

[Krout0n]

Hi, I'm struggling to design physical-server-architecture of my application using Secretflow. So I was looking at the documentation, noticed PYU can take 2 arguments but no usage is described and used in this repository.

I will ask again if it is feasible when I have a sample architecture diagram, but for now I would be glad to know the specific use/example of the second argument of PYU. Maybe this will help me think about my architecture.

Have a happy new year!

[zhouaihui]

Hi, @Krout0n , this is really a good question!

PYU takes party as the first arg and node as second arg in construction. And node defaults to empty by now.

node is designed to identify two different PYUs of the same party.
The __str__ and __repr__ of PYU is party_node , while __eq__ and __hash__ of PYU are based on str(PYU). So we can tell the difference of two PYUs of the same party, e.g., one PYU is tagged with node 1 and the other PYU is tagged with node 2.

As you can see, the built-in algorithms of SecretFlow haven't encountered a situation where two PYUs with same party were built.
The main reason is the computing of one party occurs in one PYU by now. Maybe node will be used when computing of one party needs to be extended to multiple machines. But i can not promise this and maybe the node arg will be changed in the future.

What i am interested is your struggle about physical-server-architecture. Could you share more infos?

[Krout0n]

Hi, @zhouaihui ! Thank you for your answer. This explanation is easy to understand.

So we can tell the difference of two PYUs of the same party, e.g., one PYU is tagged with node 1 and the other PYU is tagged with node 2.

BTW, thanks again for your interest about server-architecture.

What i am interested is your struggle about physical-server-architecture. Could you share more infos?

I have reconsidered once from the answers you sent me. What I really wanted to know is how to separate SPU and PYU on different machines, I know that in the SPU cluster definition I can specify the IP address and port of the SPU server, but I do not know how to separate them. Please see the attached image, which shows the case of running psi_join_csv with separate SPU and PYU. I expect the following steps to be performed

• When I run psi_join_csv, each PYU of Alice and Bob reads the local CSV.
• Send the read CSV to the respective SPU servers
• Run psi_join_csv.
• Save the results to the machines with Alice and Bob's PYUs, respectively.

I've lost the purpose of this discussion's original question...should I start a new one?　Thank you for your always thoughtful and helpful answers. I really appreciate it!

[6fj]

Hi @Krout0n

The short answer is for psi_join_csv, spu device will handle all the stuff - including csv IO and psi computation.

[6fj]

But generally, well, PYU devices would be responsible for local IO and SPU device would do MPC computation. So your understanding is correct and solid in most cases.

[Krout0n]

Hi @6fj

The short answer is for psi_join_csv, spu device will handle all the stuff - including csv IO and psi computation.

Can I separate SPU and PYU if other calculations, psi_join_df or other instructions done by SPU?

[6fj]

Hi @Krout0n

Yes, you could do that. But I have to clarify that underneath, psi_join_df is calling psi_join_csv:

secretflow/secretflow/device/device/spu.py

Lines 465 to 524 in 72aac5f

	def psi_join_df(
	self,
	key: Union[str, List[str]],
	data: pd.DataFrame,
	receiver: str,
	join_party: str,
	protocol='KKRT_PSI_2PC',
	precheck_input=True,
	bucket_size=1 << 20,
	curve_type="CURVE_25519",
	):
	"""Private set intersection with DataFrame.
	Examples:
	>>> spu = sf.SPU(utils.cluster_def)
	>>> spu.psi_join_df(['c1', 'c2'], [df_alice, df_bob], 'alice', 'alice')
	Args:
	key (str, List[str]): Column(s) used to join.
	data (pd.DataFrame): DataFrame to be joined.
	receiver (str): Which party can get joined data, others will get None.
	join_party (str): party joined data
	protocol (str): PSI protocol, See spu.psi.PsiType.
	precheck_input (bool): Whether to check input data before join.
	bucket_size (int): Specified the hash bucket size used in psi. Larger values consume more memory.
	curve_type (str): curve for ecdh psi
	Returns:
	pd.DataFrame or None: joined DataFrame.
	"""
	# save key dataframe to temp file for streaming psi
	data_dir = f'.data/{self.rank}-{uuid.uuid4()}'
	os.makedirs(data_dir, exist_ok=True)
	input_path, output_path = (
	f'{data_dir}/psi-input.csv',
	f'{data_dir}/psi-output.csv',
	)
	data.to_csv(input_path, index=False)
	try:
	report = self.psi_join_csv(
	key,
	input_path,
	output_path,
	receiver,
	join_party,
	protocol,
	precheck_input,
	bucket_size,
	curve_type,
	)
	if report['intersection_count'] == -1:
	# can not get result, return None
	return None
	else:
	# load result dataframe from temp file
	return pd.read_csv(output_path)
	finally:
	shutil.rmtree(data_dir, ignore_errors=True)

.