firebase-tools icon indicating copy to clipboard operation
firebase-tools copied to clipboard

[FR] Support for scheduled functions in emulator

Open Elyx0 opened this issue 5 years ago • 42 comments

[REQUIRED] Environment info

firebase-tools: 7.15.0 & 7.15.1

Platform: macOS 18.7.0 Darwin Kernel Version 18.7.0:

[REQUIRED] Test case

export const pubsubScheduled = functions.pubsub
  .schedule('every 1 minutes')
  .onRun(context => {
    console.log('Resource', JSON.stringify(context.resource));
    return true;
  });

[REQUIRED] Steps to reproduce

firebase emulators firebase emulators:start ✔ functions[pubsubScheduled]: pubsub function initialized.

[REQUIRED] Expected behavior

Logging every 1 minute in the console

[REQUIRED] Actual behavior

No output

Links: https://github.com/firebase/firebase-tools/commit/b6fa88aa166bddc8969cbed7bd3b35392e307dd5

https://github.com/firebase/firebase-tools/commit/4dc64dac1d54f9e863fa2a99769bca447320940f

https://github.com/firebase/firebase-tools/pull/1543#issuecomment-600763141

Elyx0 avatar Mar 18 '20 17:03 Elyx0

@Elyx0 I think we maybe misled with how we represented https://github.com/firebase/firebase-tools/pull/2011

It lets those functions be loaded into the emulator but doesn't actually trigger them on a schedule. Instead you'd have to manually trigger them using a Pub/Sub message.

This was mostly meant for use within functions:shell but I can definitely see how this is confusing and we need to clarify it.

samtstern avatar Mar 18 '20 18:03 samtstern

Thanks for clarifying @samtstern I'll wrap them in a setInterval in the global scope if it detects the emulator

Elyx0 avatar Mar 23 '20 16:03 Elyx0

I have the five lines of SetInterval code running fine to get around the absence of this, but it clutters up my index file with redundant .onPublish and .onRun calls to the same code, and the workaround might up the cold start time a bit for the non-PubSub functions, so it would be great if this feature could get built thanks!

spicemix avatar May 09 '21 02:05 spicemix

@spicemix Thanks for the suggestion! To keep the issues focussed, could you please file a separate feature request for this? We are unable to promise any timeline for this, but if others also have this request, adding a +1 on that new feature request can help us prioritize adding this to the roadmap.

weixifan avatar May 19 '21 01:05 weixifan

For internal reference: the feature request is b/188590642.

weixifan avatar May 19 '21 01:05 weixifan

I would just re-open this until it's actually addressed, I only explained some motivation for fixing this issue.

spicemix avatar May 19 '21 01:05 spicemix

@spicemix Sure, my thought was that it might be easier for users to discover the FR and +1 on a new FR rather than having to read through this bug report and realize there is a related FR. I'll change the title to make this clear.

weixifan avatar May 19 '21 15:05 weixifan

Ah, Sam has changed the title already, thanks!

weixifan avatar May 19 '21 15:05 weixifan

@samtstern could you confirm that there's still no way to put breakpoint on pubsub functions by either invoking them manually or on interval basis?

Only way I can invoke pubub is via firebase functions:shell but it still doesn't support putting breakpoint in VSCode :(

Is there any solution by which I can put breakpoints on pubsub functions and efficiently debug them locally on VSCode?

GorvGoyl avatar May 20 '21 17:05 GorvGoyl

@GorvGoyl if you create a separate HTTP function which uses the @google-cloud/pubsub SDK to send PubSub messages to trigger your functions, you can debug them like any other.

I normally use a trick like this to expose a special testing webhook only when emulating:

const { PubSub } = require("@google-cloud/pubsub");
const pubsubClient = new PubSub({  projectId: process.env.GCLOUD_PROJECT });

if (process.env.FUNCTIONS_EMULATOR === "true") {
  exports.firePubsub = functions.https.onRequest(async (request, response) => {
    const publisher = pubsubClient.topic("YOUR_TOPIC_HERE").publisher();
    const data = { 
      // ... 
    };
    const dataBuffer = Buffer.from(JSON.stringify(data));
    await publisher.publish(dataBuffer);

    response.status(200).send("Fired PubSub");
  });
}

If the PubSub emulator is running then the pubsubClient will automatically communicate with it and then the Functions emulator will run any PubSub functions that trigger on your topic.

samtstern avatar May 20 '21 18:05 samtstern

Not sure if it helps, but you can use firebase functions:shell --inspect-functions 9229 and then on VSCode use attach to node process to debug fns interactively.

johnnyasantoss avatar May 20 '21 18:05 johnnyasantoss

Thank you @samtstern. I think your approach is good if pubsub functions needs to be triggered naturally. And thank you @johnnyasantoss for that simple command, I can now manually trigger functions.pubsub.schedule functions and put breakpoints.

GorvGoyl avatar May 20 '21 20:05 GorvGoyl

One issue I noticed when running firebase functions:shell --inspect-functions 9229 is that I can trigger normal cloud functions from both command prompt helloWorld() or visiting url http://localhost:5001/proj/us-central1/helloWorld. However, I can only trigger scheduled functions from command prompt myScheduledFunction() but not url.

If I visit url http://localhost:5001/proj/us-central1/myScheduledFunction for my scheduled function myScheduledFunction it says Function myScheduledFunction does not exist, valid triggers are: helloWorld, myScheduledFunction-0 so, when I visit http://localhost:5001/proj/us-central1/myScheduledFunction-0 it crashes the local server with error:

TypeError: Cannot read property 'data' of undefined at processBackground (C:\Users\1gour\AppData\Roaming\npm\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:569:24) at invokeTrigger (C:\Users\1gour\AppData\Roaming\npm\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:653:19) at handleMessage (C:\Users\1gour\AppData\Roaming\npm\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:740:15) at C:\Users\1gour\AppData\Roaming\npm\node_modules\firebase-tools\lib\emulator\functionsEmulatorRuntime.js:775:20 at processTicksAndRejections (internal/process/task_queues.js:97:5)

GorvGoyl avatar May 20 '21 20:05 GorvGoyl

Hey, @GorvGoyl is there any updates on this issue?

eulier1 avatar Dec 16 '21 16:12 eulier1

any updates on supporting actual scheduled functions?

voltuer avatar Mar 23 '22 14:03 voltuer

any news about this?

mannueru avatar Apr 26 '22 01:04 mannueru

We're looking at getting staffing for this. It will probably be a few more months at least.

inlined avatar May 02 '22 21:05 inlined

Easy CSAT impact for a 20%er you like.

spicemix avatar May 03 '22 01:05 spicemix

+1 need pretty badly at the moment, having to hack around

algoflows avatar Jul 26 '22 17:07 algoflows

+1 @inlined Any news on this feature?

gerryau avatar Aug 12 '22 11:08 gerryau

As you may have heard leaks online, funding for projects at Google have gotten pretty tight. This is currently funded only for 20% work, which we have no volunteers for at the moment. I plan to make more formal pitches for 20% work and if anyone externally would like to contribute I make sure they're connected with the right people so that their work can be used right away.

The good news is that scheduled functions for Cloud Functions for Firebase v2 are coming out soon and those are simpler (they're based on HTTPS functions instead of Pub/Sub functions). It may be more feasible to just curl those functions than it was in v1.

inlined avatar Aug 15 '22 16:08 inlined

@inlined, thanks for putting up all the efforts. I'm a little confused why you would need funding if Google itself owns Firebase. Are engineers working on this project not employees at Google?

GorvGoyl avatar Aug 15 '22 17:08 GorvGoyl

Sorry, my manager-speak is showing. "Funding" means "I allocate X engineers for Y months on Z project instead of A B or C". When we don't have enough engineers to go around, we either hire more engineers or we reduce the priority of other projects. As it stands, we have more feature requests than engineers to build them; our current efforts are going into major platform improvements like Cloud Functions 2nd generation and supporting modern web frameworks like Angular and Next.js. I'm still trying to find someone to take the reins on emulator projects, but this project is currently unstaffed.

inlined avatar Aug 15 '22 23:08 inlined

Workaround

source: https://stackoverflow.com/a/69424195/1418456

use the functions shell to run the scheduled functions on a interval:

firebase functions:shell
# a shell with a "firebase >" prompt should open
setInterval(() => yourScheduledFunc(), 60000)

auto-start

if you want to start scheduled functions when you start the emulator, you can create a setupEmulator.sh file with all the intervals you wanna start, like so:

# setupEmulator.sh
setInterval(() => yourScheduledFunc(), 60000)

now you can use emulators:exec to load that up on functions:shell right after booting the emulators:

# we also cat /dev/stdin so the shell keeps listening for user input, or else it would close immediately
firebase emulators:exec "cat setupEmulator.sh /dev/stdin | firebase functions:shell"

finally, put the command above on a package.json script and you're set:

// package.json
"scripts": {
   "emulator": "firebase emulators:exec 'cat setupEmulator.sh /dev/stdin | firebase functions:shell'"
}

run it with yarn emulator or npm run emulator

volkyeth avatar Sep 23 '22 20:09 volkyeth

any update ?

RajendraAVerma avatar Sep 07 '23 05:09 RajendraAVerma

😬

ekalosak avatar Sep 16 '23 16:09 ekalosak

+1 please implement the feature)

Mr-Goldberg avatar Sep 26 '23 17:09 Mr-Goldberg

+1

ortegoncarlos avatar Sep 26 '23 17:09 ortegoncarlos

+1

emilio-yeshearing avatar Oct 11 '23 00:10 emilio-yeshearing

+1

johngrimsey avatar Oct 26 '23 10:10 johngrimsey