Not possible to use @hypermod/cli to run transforms located in a private NPM package, right?

[alistairwilliamtaylor]

Hello, and thank you for this wonderful project!

I'm just a junior dev trying to figure out how to set up codemods for our company's design system. I'd been playing around with setting up a script in our package which would use the jsCodeshift executable to run the transforms I'd written, but then I found hypermod and thought that could potentially save me from myself and I could instruct consumers to run something like:

npx @hypermod/cli --packages @myob/myob-widgets@9.0.0 ./src

However, our package @myob/myob-widgets is a private package, so I'm imagining that's why I'm getting the following error:

Error: Unable to locate package from Hypermod Community or NPM.
Make sure the package name "@myob/myob-widgets" is correct and try again.

Am I right in thinking that it won't work because @myob/myob-widgets is a private package? And are my options therefore to either (a) create a separate public NPM package with transforms which hypermod would be able to access, or (b) try to set up a script myself to run the transforms with jsCodeshift in our private package.

Thanks for your help, and my apologies if this question is in the wrong place / wildly off the mark

[danieldelcore]

Hey @alistairwilliamtaylor 👋 Thanks for raising an issue! I should really create a page for this on the site.

It should be possible to use hypermod for private packages, you might need to use either the --registry and/or --registryToken flags to point the CLI to your private npm registry.

https://www.codeshiftcommunity.com/docs/cli#--registry
https://www.codeshiftcommunity.com/docs/cli#--registrytoken

Assuming developers within your company are using the correct .npmrc configuration, the --registry URL should be sufficient 🤔. Could you give it a go and let me know if that solves your problem?

[alistairwilliamtaylor]

Oh thank you so much for getting back to me about this - I'll give it a go in the next few days and let you know

[danieldelcore]

Wonderful, feel free to reach out whenever! I'd love to make sure you guys can use hypermod successfully! 🙏

[alistairwilliamtaylor]

Thanks Daniel!

I've been having a go over the past few days, but sadly no success yet.

I might potentially have a bit of a lead, though? I initially tried just with --registry but haven't had any luck with that, so tried providing --registryToken as well. This produced a somewhat different result, because the CLI would get to:

⚠️  Unable to locate Hypermod package: @hypermod/mod-myob__myob-widgets
ℹ Attempting to download npm package: @myob/myob-widgets

And then the cursor just hangs and hangs on that - it never produces an error message, but it also doesn't work. I tried providing a bogus registry and/or bogus registry token and it fails immediately if either of them is wrong, which leads me to believe that when it hangs it is a somewhat encouraging sign?

I'm happy to keep playing around with this, please feel free to let me know if you have any suggestions about things I could try

[danieldelcore]

Ah really interesting, I wonder why it's hanging like that 🤔 Does the npm package have a large bundlesize? It could be hanging while it downloads and maybe fail silently. Could you share some logs if there are any?

[alistairwilliamtaylor]

Initially I was cancelling the command because it was hanging for so long, but if I leave it for 5-6 minutes I eventually get:

<--- Last few GCs --->

[53233:0x130008000]   509554 ms: Scavenge 3931.4 (4128.2) -> 3919.4 (4128.2) MB, 6.8 / 0.0 ms  (average mu = 0.227, current mu = 0.136) task
[53233:0x130008000]   509623 ms: Scavenge 3931.7 (4128.2) -> 3920.6 (4129.2) MB, 8.9 / 0.0 ms  (average mu = 0.227, current mu = 0.136) task
[53233:0x130008000]   512496 ms: Mark-sweep 3933.4 (4129.2) -> 3919.7 (4130.0) MB, 2807.4 / 0.0 ms  (average mu = 0.200, current mu = 0.172) task scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x102c82984 node::Abort() [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 2: 0x102c82b74 node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 3: 0x102dc56e8 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 4: 0x102dc56ac v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 5: 0x102f474d4 v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 6: 0x102f4ad30 v8::internal::Heap::CollectSharedGarbage(v8::internal::GarbageCollectionReason) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 7: 0x102f47c98 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 8: 0x102f455bc v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
 9: 0x102fc5bcc v8::internal::ScavengeJob::Task::RunInternal() [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
10: 0x102cde5d8 node::PerIsolatePlatformData::RunForegroundTask(std::__1::unique_ptr<v8::Task, std::__1::default_delete<v8::Task>>) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
11: 0x102cdd2f4 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
12: 0x1034f5244 uv__async_io [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
13: 0x103506de4 uv__io_poll [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
14: 0x1034f56d4 uv_run [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
15: 0x102bc96c0 node::SpinEventLoop(node::Environment*) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
16: 0x102cbc800 node::NodeMainInstance::Run(int*, node::Environment*) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
17: 0x102cbc4ec node::NodeMainInstance::Run() [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
18: 0x102c54370 node::Start(int, char**) [/Users/Alistair.Taylor/.nvm/versions/node/v16.20.2/bin/node]
19: 0x19bfeff28 start [/usr/lib/dyld]
[1]    53232 abort      npx @hypermod/cli --registry https://{PRIVATE_REGISTRY_HERE}/npm/ --registryToken

[danieldelcore]

Ah I see, the package loader we use has some drawbacks. I suspect the bundlesize is too large which is causing it to run out of memory. I have implemented a more robust loader behind the [--experimentalLoader](https://github.com/hypermod-io/hypermod-community/blob/2b9179a34efcd9d1edcdbead5b1fd6c839f72d75/packages/cli/src/index.ts#L70C5-L70C5) flag. It should be able to handle larger packages but doesn't yet have support for --registryToken` 😭 I'll have to update it and see if that fixes your issue.

Is the package you're deploying you're codemods with quite large?

[alistairwilliamtaylor]

Yeah it is quite large - it has all of our components for the design system in it. Sorry about that! I'm not opposed to creating a separate package for the hypermod codemods for now, so that I can start playing around more with hypermod. I'll try to have a go this week.

Thanks for all of your support with this! 😄

[danieldelcore]

No worries at all! Feel free to reach out if you run into any more issues, I'd be more than happy to help!

Yeah, a standalone package in this case would be the best way to go!
I recommend having a look at this guide: https://www.codeshiftcommunity.com/docs/external-packages 😄