Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to correctly describe a pointer and a pointer to a pointer? #92

Closed
yndtrud opened this issue Jan 20, 2025 · 27 comments
Closed

How to correctly describe a pointer and a pointer to a pointer? #92

yndtrud opened this issue Jan 20, 2025 · 27 comments

Comments

@yndtrud
Copy link

yndtrud commented Jan 20, 2025

Current ffi-rs version

type node_modules\ffi-rs\package.json
{
  "name": "ffi-rs",
  "version": "1.2.3",
  "main": "index.js",
  "types": "index.d.ts",
  "description": "A module written in Rust and N-API provides interface (FFI) features for Node.js",
  "napi": {
    "name": "ffi-rs",
    "triples": {
      "additional": [
        "aarch64-apple-darwin",
        "aarch64-unknown-linux-gnu",
        "aarch64-unknown-linux-musl",
        "arm-unknown-linux-gnueabihf",
        "i686-pc-windows-msvc",
        "x86_64-unknown-linux-musl",
        "aarch64-pc-windows-msvc"
      ]
    }
  },
  "author": "zhangyuang",
  "homepage": "https://github.com/zhangyuang/node-ffi-rs#readme",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/zhangyuang/node-ffi-rs.git"
  },
  "keywords": [
    "ffi",
    "rust",
    "node.js",
    "napi"
  ],
  "files": [
    "index.js",
    "index.d.ts",
    "README.md"
  ],
  "license": "MIT",
  "dependencies": {},
  "devDependencies": {
    "@napi-rs/cli": "^2.15.2",
    "@types/node": "^20.8.7",
    "benny": "^3.7.1",
    "conventional-changelog-cli": "^4.1.0",
    "esno": "^4.0.0",
    "ffi-napi": "^4.0.3",
    "koa": "^2.14.2",
    "shelljs": "^0.8.5",
    "typescript": "^5.4.5"
  },
  "scripts": {
    "artifacts": "napi artifacts",
    "build:c": "node scripts/compile.js",
    "build:dev": "env=development node scripts/build.js",
    "build": "node scripts/build.js",
    "publish:npm": "node scripts/publish.js",
    "test": "esno ./tests/index.ts",
    "changelog": "conventional-changelog -p angular -i CHANGELOG.md -s -r 0 && git add . && git commit -m \"docs: update changelog.md\" && git push origin master",
    "pub": "npm version patch && git push origin master --tags && npm run changelog",
    "pub:alpha": "npm version prerelease --preid=alpha && git push origin master --tags"
  },
  "optionalDependencies": {
    "@yuuang/ffi-rs-darwin-arm64": "1.2.3",
    "@yuuang/ffi-rs-darwin-x64": "1.2.3",
    "@yuuang/ffi-rs-linux-arm-gnueabihf": "1.2.3",
    "@yuuang/ffi-rs-linux-arm64-gnu": "1.2.3",
    "@yuuang/ffi-rs-linux-arm64-musl": "1.2.3",
    "@yuuang/ffi-rs-linux-x64-gnu": "1.2.3",
    "@yuuang/ffi-rs-linux-x64-musl": "1.2.3",
    "@yuuang/ffi-rs-win32-arm64-msvc": "1.2.3",
    "@yuuang/ffi-rs-win32-ia32-msvc": "1.2.3",
    "@yuuang/ffi-rs-win32-x64-msvc": "1.2.3"
  }
}

dir node_modules\@yuuang\
 Volume in drive D is Dev
 Volume Serial Number is 1111-1111

 Directory of node_modules\@yuuang

20.01.2025  14:25    <DIR>          .
20.01.2025  14:25    <DIR>          ..
20.01.2025  14:23    <DIR>          ffi-rs-win32-ia32-msvc
               0 File(s)              0 bytes
               3 Dir(s)  48 303 276 847 104 bytes free

Current Node.js arch

node --version
v18.20.5

node -e "console.log(process.arch, process.platform)"
ia32 win32

Descibe your problem in detail

I read the description of the library but could not call some WinAPI functions. Perhaps I didn't understand correctly how to use the library. I would really like you to explain it to me.

There are a couple of functions:

  1. MemAlloc - allocates memory and returns a pointer to the allocated memory to the calling program. To clarify, the function will allocate a buffer and fill this byffer with useful data.
  2. MemFree - a pointer to a buffer is passed to the function and the function frees this memory

Pseudo example of Windows x32-x86 dll module:

LONG MemAlloc(GUID* pGUID, LPWSTR* pszBuffer, LPDWORD pdwCount = NULL)
{
 ::UuidCreate(pGUID);
 pdwCount = 100;
 *pszBuffer= (LPWSTR)::HeapAlloc(::GetProcessHeap(), HEAP_ZERO_MEMORY, pdwCount * sizeof(WCHAR));
  return ERROR_SUCCESS /* 0 */;
}


VOID MemFree(LPWSTR pBuffer)
{
 ::HeapFree(::GetProcessHeap(), 0x00, pBuffer);
}

// C++ Usage
GUID giud = {0};
LPWSTR pszBuffer = NULL;
DWORD dwCount = 0;

MemAlloc(&guid, &pszBuffer, &dwCount)
MemFree(pszBuffer);

How to call such functions correctly? I don't understand how to describe the parameters for

  1. How to import functions correctly?
  2. GUID*: pointer to GUID
  3. &LPWSTR -> char** : the library allocates memory and copies text to this buffer (MemAlloc)
  4. LPWSTR -> char*: Library free memory (MemFree)
  5. LPDWORD -> unsigned long*: the library returns the buffer size to this variable
  6. NULL pointer?

My implementation in Node

const winapi = define({
  MemAlloc: {
            library: WinApi32dll,
            retType: DataType.I32,
            paramsType: [ DataType.External, DataType.External, DataType.External ] 
  }, 
  MemFree: {
            library: WinApi32dll,
            retType: DataType.Void,
            paramsType: [ DataType.External ] 
  }
})

To describe the GUID in NODE I used this monster construction.
It works. Next, I convert it from binary to text form and everything is ok. Did I choose the right way or not?

let GUID = new Buffer.alloc(16).fill(0);
const ptrGUID = unwrapPointer(createPointer({ paramsType: [DataType.U8Array], paramsValue: [ GUID ] }))[0];
winapi.MemAlloc([ptrGUID, SecondParamLPWSTR, ThirdParamLPDWORD]);

For a pointer to a DWORD I can use a pointer to an Int32, but it doesn't work!

let dwCount = 13; /* some garbage*/
const ptrdwCount  = unwrapPointer(createPointer({ paramsType: [DataType.I32], paramsValue: [ dwCount ] }))[0];
winapi.MemAlloc([FirstParamPGUID, SecondParamLPWSTR, ptrdwCount)] );

>> Process exited with code 3221225477 (0xC0000005)

What would be the correct way to send NULL in WinAPI?

#define NULL = 0 // nullptr but how in Node.JS?

I can’t understand how to describe pointer to pointer
LPWSTR* -> char** - the most important parameter for both functions

@zhangyuang
Copy link
Owner

Please format your text with markdown syntax, i can't read it friendly.

If you want to create a pointer point to i32, just use createPointer without unwrapPointer

@yndtrud
Copy link
Author

yndtrud commented Jan 21, 2025

Please format your text with markdown syntax, i can't read it friendly.

If you want to create a pointer point to i32, just use createPointer without unwrapPointer

Sorry. I corrected the formatting and tried to describe it in more detail
Please look again :)

@zhangyuang
Copy link
Owner

You can refer to other issues to realize how to call win API by ffi-rs

@zhangyuang
Copy link
Owner

I advise you to figure out the parameters meaning of c.
For example, GUID* pGUID is a uuid type pointer, you can create a pointer to store the true value

typedef struct _GUID {
    unsigned long  Data1;    // 32位整数
    unsigned short Data2;    // 16位整数
    unsigned short Data3;    // 16位整数
    unsigned char  Data4[8]; // 8个字节的数组
} GUID;

const uuidPtr = createPointer({
paramsType: [{
Data1: DataType.i32
Data4:  arrayConstructor({
    type: DataType.U8Array,
    length:8
    ffiTypeTag: DataType.StackArray
  })
},
paramsValue: [{
Data1: 0,
Data2: 0,
Data4: Buffer.from(new Array(8).fill(0)),

}]
]
})

@yndtrud
Copy link
Author

yndtrud commented Jan 21, 2025

Please format your text with markdown syntax, i can't read it friendly.

If you want to create a pointer point to i32, just use createPointer without unwrapPointer

const winapi = define({
  MemAlloc: {
            library: WinApi32dll,
            retType: DataType.I32,
            paramsType: [ DataType.External, DataType.External, DataType.External ] 
  }
})

let dwCount = 13; /* some garbage*/
const ptrdwCount  = createPointer({ paramsType: [DataType.I32], paramsValue: [ dwCount ] }));
winapi.MemAlloc([FirstParamPGUID, SecondParamLPWSTR, ptrdwCount)] );

Without unwarp: Uncaught Error Error: expect External, got: Object

@zhangyuang
Copy link
Owner

createPointer will return an array of pointer, you should use index 0 to get the pointer element. You can write it with typescript for type hints

@yndtrud
Copy link
Author

yndtrud commented Jan 22, 2025

createPointer will return an array of pointer, you should use index 0 to get the pointer element. You can write it with typescript for type hints

Yes, it compiles but doesn't work as expected:

const winapi = define({
  MemAlloc: {
            library: WinApi32dll,
            retType: DataType.I32,
            paramsType: [ DataType.External, DataType.External, DataType.External ] 
  }
})

let dwCount = 0; 
const dwCountPtr  = createPointer( { paramsType: [DataType.I32], paramsValue: [ dwCount ] } )[0];
winapi.MemAlloc( [ FirstParamPGUID, SecondParamLPWSTR, dwCountPtr ] ); // set *dwCountPtr = 100 in C++
console.log(dwCount, dwCountPtr);

Output: 0 {}

@zhangyuang
Copy link
Owner

Use restorePointer to restore the value in the pointer

@yndtrud
Copy link
Author

yndtrud commented Jan 22, 2025

Use restorePointer to restore the value in the pointer

Final for LPDWORD. It works! Thank you.

const dwCountPtr  = createPointer( { paramsType: [DataType.I32], paramsValue: [ 0 ] } )[0];
winapi.MemAlloc( [ FirstParamPGUID, SecondParamLPWSTR, dwCountPtr ] ); // set *dwCountPtr = 100 in C++
let dwCount = restorePointer( { retType: [DataType.I32], paramsValue: [ dwCountPtr ] });
console.log(dwCount);

Output: 100


Question about this code: Should I free the pointer? Or should I clear the pointer only if in createPointer paramsType = DataType.External?

createPointer({ paramsType: [DataType.External]....

Often in WinAPI we use NULL (nullptr) in parameters instead of a data pointer to indicate to WinAPI that we are not using this parameter.

What is the best way to create NULL (nullptr)?

const nullptr = createPointer( { paramsType: [DataType.Void], paramsValue: [ null ] } )

@zhangyuang
Copy link
Owner

Use freePointer method when you no longer use the pointer.

Use the code below to create null pointer

// void type has already a null pointer, so ptr is a pointer point to null pointer
const ptr = createPointer( { paramsType: [DataType.Void], paramsValue: [ null ] } )

// use unwrapPointer to get the inner null pointer
const nullPtr = unwrapPointer(ptr)[0]

@yndtrud
Copy link
Author

yndtrud commented Jan 22, 2025

The most significant problem.

I have greatly simplified the example and left only the essence of the problem.

C++

// WinApi32.dll

// Allocate 100 chars (bytes) buffer and fill 99 chars with 'A', return POINTER to allocated buffer
void MemAlloc(char** ppBuffer)
{
 *ppBuffer= (char*)::HeapAlloc(::GetProcessHeap(), HEAP_ZERO_MEMORY, 100); // Allocate 100 (bytes) buffer and save pointer
  memset ((void*)*ppBuffer, 65 /* 'A' */, 99 ); // fill 99 chars with 'A'
}

// Free allocated buffer
void MemFree(char* pBuffer)
{
 ::HeapFree(::GetProcessHeap(), 0x00, pBuffer);
}

// C++ Usage
int main(void)
{
  char* pBuffer = nullptr;
  
  MemAlloc(&pBuffer);
  printf("%s", pBuffer); // -> AAAAAAAAAAAAAAAAAAAAAAAAAAA....
  MemFree(pBuffer);
 
  return 0;
}

JS

I don't know how to correctly describe import.
I don't understand how to call these functions from JS.

const winapi = define({
  MemAlloc: {
            library: WinApi32dll,
            retType: DataType.Void,
            paramsType: [ DataType.External] 
  }, 
  MemFree: {
            library: WinApi32dll,
            retType: DataType.Void,
            paramsType: [ DataType.External ] 
  }
})

winapi.MemAlloc(????);
winapi.MemFree(????);

Sorry for my persistence but totally stuck at this point in code.

@zhangyuang
Copy link
Owner

You can find the same type definition in test cases and call examples

char** is a string array type

@yndtrud
Copy link
Author

yndtrud commented Jan 22, 2025

You can find the same type definition in test cases and call examples

char** is a string array type

I looked at the code provided and did not find a solution.

In your examples there is a size of the returned data, you create arrays based on this size or get ** in retType of DataType.External.
In my case, the library returns a pointer to a buffer in a variable that I specified, in which I can find data, the size is not known to me and is not needed, since it will simply return a zero-terminated buffer to me.

  char* pBuffer = nullptr;  
  MemAlloc(&pBuffer);
  printf("%s", pBuffer); // -> AAAAAAAAAAAAAAAAAAAAAAAAAAA....
  MemFree(pBuffer);`

@zhangyuang
Copy link
Owner

zhangyuang commented Jan 22, 2025 via email

@yndtrud
Copy link
Author

yndtrud commented Jan 22, 2025

you can't restore an array without specify length safety,or you can restore it with a big size

发自我的iPhone

Let's do it differently, without text (char) data.

The buffer will be a void type:

void* pBuffer = nullptr;

I want to get a pointer to it (it is allocated in C++)

MemAlloc(&pBuffer);

and I want to move through it until I find the first zero, outputting all the data to the console.

At the end, I'll call flushing this buffer using C++.

MemFree(pBuffer);

How to get a direct pointer to allocated C++ memory on the heap?

const voidPtr = createPointer({ paramsType: [DataType.RefVoid], paramsValue: [ 0 /* init value */ ] }); // <-- C++  void* pBuffer = nullptr;
let pVoid = unwrapPointer(voidPtr)[0]; // &pBuffer 
winapi.MemAlloc(pVoid); // C++ MemAlloc(&pBuffer);

let text = Buffer.from(refPointer(voidPtr)[0], 0, 100 /* 100 for simplicity */) /* function not found in ffi-rs*/);
console.log(text.toString());

winapi.MemFree(pVoid); // C++ MemFree(pBuffer);

@zhangyuang
Copy link
Owner

zhangyuang commented Jan 22, 2025 via email

@yndtrud
Copy link
Author

yndtrud commented Jan 22, 2025

create DataType.u8array pointer with an enough size and fill it in c environment

发自我的iPhone

Do I understand correctly that your library cannot work with pointer to pointer as argument to functions in C++ ?

func2(int **x);

Example:

void func1(int *);
void func2(int **);

int i = 100;
int *ptr = &i;

// when you want value of ptr should remain unchanged, but you want to change only value of i, use,
func1(int *)

// when you want value of ptr should change. i.e, it should point to some other memory other than i, use,
func2(int **);

@zhangyuang
Copy link
Owner

Use funcConstructor to create a function pointer, then you can use createPointer method multiply to create arbitrary multiple pointer

@yndtrud
Copy link
Author

yndtrud commented Jan 22, 2025

Use funcConstructor to create a function pointer, then you can use createPointer method multiply to create arbitrary multiple pointer

Finally. My fully working version.
It works as I need, tell me, did I write everything correctly, the ffi-rs JS logic? No JS or Rust memory leaks?

In C++ everything works correctly.

const voidPtr = createPointer({ paramsType: [DataType.Void], paramsValue: [ null ] }); // void* pBuffer = nullptr;
const stringPtr = wrapPointer(voidPtr)[0]; // &pBuffer 
winapi.MemAlloc(stringPtr); // MemAlloc(&pBuffer)
console.log(restorePointer( { retType: [DataType.WString] , paramsValue: [ stringPtr ] } ), // printf('%s', (wchar*)pBuffer);
winapi.MemFree(stringPtr ); // MemFree(pBuffer);
freePointer({ paramsType: [DataType.Void], paramsValue: voidPtr, pointerType: PointerType.RsPointer } );

@yndtrud
Copy link
Author

yndtrud commented Jan 24, 2025

The process crashes when I use freePointer. What is my mistake?

// Process exited with code 3221226505

let ctxData = new Buffer.alloc(16).fill(0);
const ctxPtr = createPointer({ paramsType: [DataType.U8Array], paramsValue: [ ctxData] });
freePointer({ paramsType: [DataType.U8Array], paramsValue: ctxPtr, pointerType: PointerType.RsPointer } );      

@zhangyuang
Copy link
Owner

 let ctxData = Buffer.alloc(16).fill(0);
  const ctxPtr = createPointer({ paramsType: [arrayConstructor({ type: DataType.U8Array, length: 16 })], paramsValue: [ctxData] });
  freePointer({ paramsType: [arrayConstructor({ type: DataType.U8Array, length: 16 })], paramsValue: ctxPtr, pointerType: PointerType.RsPointer });

@yndtrud
Copy link
Author

yndtrud commented Jan 24, 2025

[arrayConstructor({ type: DataType.U8Array, length: 16 })]

Thank you, your code works, but I don’t understand what’s wrong with my version?
Is the implementation wrong?

@zhangyuang
Copy link
Owner

Use arrayConstructor to create array type instead of DataType.array

@yndtrud
Copy link
Author

yndtrud commented Jan 24, 2025

I guess this is my last question. You're probably already tired of me.
How to make such a structure?

typedef struct tagSQLCONNECTINFO {
	DWORD	cbSize;

	WCHAR	szServerName[261];
	WCHAR	szDataBaseName[129];

	BOOL	bIntegratedAuthentication;

	WCHAR	szLogin[129];
	WCHAR	szPassword[129];
}SQLCONNECTINFO;

My attempts are in JS but I don't understand if my approach is correct

// ffi-rs type
  const winapi = sqlconnectioninfoType : {
    cbSize: DataType.I32, 

    szServerName : arrayConstructor({ type: DataType.U8Array, length: 261*2, ffiTypeTag: DataType.StackArray }),
    szDataBaseName: arrayConstructor({ type: DataType.U8Array, length: 129*2, ffiTypeTag: DataType.StackArray }),

    bIntegratedAuthentication :  DataType.I32,

    szLogin : arrayConstructor({ type: DataType.U8Array, length: 129*2, ffiTypeTag: DataType.StackArray }),
    szPassword : arrayConstructor({ type: DataType.U8Array, length: 129*2, ffiTypeTag: DataType.StackArray })
  }

// JS user data
const sqlconnectinfo = { 
  server: '(local)', 
  database: 'model', 
  intauth: false, 
  login : 'sqluser', 
  password: 'sqlpass'
 };

// ffi-rs pointer
const _sqlconnectinfo_ptr = createPointer({ paramsType: [ winapi.sqlconnectioninfoType ], paramsValue: [ {
  cbSize: winapi.sqlconnectioninfoSize.v1, /* i know size */

  szServerName: Buffer.from(sqlconnectinfo.server + '\0', 'ucs2'),
  szDataBaseName: Buffer.from(sqlconnectinfo.database + '\0', 'ucs2'),

  bIntegratedAuthentication: sqlconnectinfo.intauth ? 1 : 0,

  szLogin: Buffer.from(sqlconnectinfo.login + '\0', 'ucs2'),
  szPassword: Buffer.from(sqlconnectinfo.password + '\0', 'ucs2')
}] });

// raw pointer
const _pSqlconnectinfo = unwrapPointer(_sqlconnectinfo_ptr)[0];

// Call
winapi.lib.SqlConnect(_pSqlconnectinfo );

freePointer({ paramsType: [winapi.sqlconnectioninfoType], paramsValue: _sqlconnectinfo_ptr, pointerType: PointerType.RsPointer }); 

@zhangyuang
Copy link
Owner

Support DataType.I16Array in [email protected]

Use i16Array to represent wchar array in windows, i32Array in Unix platform

 const info = {
      cbSize: 1304,
      szServerName: [...Buffer.from("localhost")],
    }

    load({
      library: "libsum",
      funcName: "printSQLCONNECTINFO",
      retType: DataType.Void,
      paramsType: [{
        cbSize: DataType.I64,
        szServerName: arrayConstructor({
          type: DataType.I16Array,
          length: 261,
          ffiTypeTag: DataType.StackArray
        }),
      }],
      paramsValue: [info]
    })

@yndtrud
Copy link
Author

yndtrud commented Jan 27, 2025

Support DataType.I16Array in [email protected]

Use i16Array to represent wchar array in windows, i32Array in Unix platform

 const info = {
      cbSize: 1304,
      szServerName: [...Buffer.from("localhost")],
    }

    load({
      library: "libsum",
      funcName: "printSQLCONNECTINFO",
      retType: DataType.Void,
      paramsType: [{
        cbSize: DataType.I64,
        szServerName: arrayConstructor({
          type: DataType.I16Array,
          length: 261,
          ffiTypeTag: DataType.StackArray
        }),
      }],
      paramsValue: [info]
    })

If there is a way to initialize an array with default value?
In your example there is no 0 (String Zero Terminator) at the end of the string, this is a problem for C++. Initialization would help not to add + '\0' to each line at szServerName: [...Buffer.from("localhost")+'\0'],

@zhangyuang
Copy link
Owner

`szServerName: [...Buffer.from("localhost\0")],`

@yndtrud yndtrud closed this as completed Feb 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants