Skip to main content

When does a pg.pool call return?

I have a series of database calls that I need to execute in serial, not in parallel. Despite using the execute_sequentially promise_factory pattern described at: https://pouchdb.com/2015/05/18/we-have-a-problem-with-promises.html

The back-end believes the promises to make each DB call are running in serial, as they should be.

The database shows that multiple queries are starting before the first one finishes; this is a problem.

So I think the DB is telling the back end it can continue before it finishes running the query. But I've struck out looking for documentation to explain that timing. Does the DB wait for the query to finish before it tells the back end that it can continue, or not?

More details:

Each function call stacks "decks" of "cards" atop the existing pile. If the function runs in serial, everything should be correct. When they run in parallel, multiple cards end up at the same height, and that's an error condition.

For example, if I have 3 decks of 2 cards each, I want to stack up with one card at each height, 0,1..,3,4. Instead I end up with multiple cards at height 0, and multiple cards at height 1. Each query checks the current height of the combined deck, gets the same answer, and starts stacking up its cards from there, instead of waiting for the previous SQL function to complete, which should change the height of the new top card.

Logging on the back end server where that code is running claims that the calls are indeed executing one at a time. I have a log line at the start of each, and at the end of each, and the logs always say they're running:

start 1
end 1
start 2
end 2
start 3
end 3

...as they should be.

However, as I've added logging code into the plpgsql function that they're calling, the mess is revealed. Log lines on the db side look more like:

start stacking deck 1
start stacking deck 2
get_height helper function start 1 
get_height helper function end 1
start stacking deck 3
end stacking deck 1
end stacking deck 2
get_height helper function start 2
get_height helper function end 2
end stacking deck 3

This leads me to suspect that the DB is reporting to the back end server that it's done, as soon as it receives the query, not when the query has in fact completed running.

P.S. It's also suspicious that the helper function on the plpgsql side isn't running once for each time the main function runs. I'm guessing that it's keeping a cached answer because it's not until a query completes that the answer would be invalidated. If I'm right, that should clear up on its own once the function calls go from parallel to serial.

P.P.S Here's a stripped down version of the plpgsql function:

CREATE FUNCTION stack_one_deck(
  game_id games.id%type,
  deck_id decks.id%type,
RETURNS void
LANGUAGE plpgsql
AS
$$
BEGIN
  RAISE LOG 'Starting stack deck with game_id % and deck_id %', game_id, deck_id;

  WITH gdh AS (
    SELECT get_deck_height(game_id)
  )
  INSERT INTO
    game_cards
    (game_id, card_id, height)
  SELECT
    game_id, card_id, (SELECT get_deck_height FROM gdh)+ordinality+1
  FROM
    deck_cards
  WHERE
    id = deck_id;

  RAISE LOG 'Ending stack scenario deck with game_id % and deck_id %', game_id, deck_id;

  END;
$$;

(Errors likely crept in when simplifying the code for the sake of readability.)

The code on the back end that creates the promise factories and executes each one in turn is substantially more complicated than the example at https://pouchdb.com/2015/05/18/we-have-a-problem-with-promises.html for multiple reasons (e.g. the call needs parameters) so forgive me for just quoting the much-simpler example from that page here:

function executeSequentially(promiseFactories) {
  var result = Promise.resolve();
  promiseFactories.forEach(function (promiseFactory) {
    result = result.then(promiseFactory);
  });
  return result;
}

function myPromiseFactory() {
  return somethingThatCreatesAPromise();
}
Via Active questions tagged javascript - Stack Overflow https://ift.tt/lUEqm3v

Comments

Popular posts from this blog

ValueError: X has 10 features, but LinearRegression is expecting 1 features as input

So, I am trying to predict the model but its throwing error like it has 10 features but it expacts only 1. So I am confused can anyone help me with it? more importantly its not working for me when my friend runs it. It works perfectly fine dose anyone know the reason about it? cv = KFold(n_splits = 10) all_loss = [] for i in range(9): # 1st for loop over polynomial orders poly_order = i X_train = make_polynomial(x, poly_order) loss_at_order = [] # initiate a set to collect loss for CV for train_index, test_index in cv.split(X_train): print('TRAIN:', train_index, 'TEST:', test_index) X_train_cv, X_test_cv = X_train[train_index], X_test[test_index] t_train_cv, t_test_cv = t[train_index], t[test_index] reg.fit(X_train_cv, t_train_cv) loss_at_order.append(np.mean((t_test_cv - reg.predict(X_test_cv))**2)) # collect loss at fold all_loss.append(np.mean(loss_at_order)) # collect loss at order plt.plot(np.log(al...

Sorting large arrays of big numeric stings

I was solving bigSorting() problem from hackerrank: Consider an array of numeric strings where each string is a positive number with anywhere from to digits. Sort the array's elements in non-decreasing, or ascending order of their integer values and return the sorted array. I know it works as follows: def bigSorting(unsorted): return sorted(unsorted, key=int) But I didnt guess this approach earlier. Initially I tried below: def bigSorting(unsorted): int_unsorted = [int(i) for i in unsorted] int_sorted = sorted(int_unsorted) return [str(i) for i in int_sorted] However, for some of the test cases, it was showing time limit exceeded. Why is it so? PS: I dont know exactly what those test cases were as hacker rank does not reveal all test cases. source https://stackoverflow.com/questions/73007397/sorting-large-arrays-of-big-numeric-stings

How to load Javascript with imported modules?

I am trying to import modules from tensorflowjs, and below is my code. test.html <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>Document</title </head> <body> <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@2.0.0/dist/tf.min.js"></script> <script type="module" src="./test.js"></script> </body> </html> test.js import * as tf from "./node_modules/@tensorflow/tfjs"; import {loadGraphModel} from "./node_modules/@tensorflow/tfjs-converter"; const MODEL_URL = './model.json'; const model = await loadGraphModel(MODEL_URL); const cat = document.getElementById('cat'); model.execute(tf.browser.fromPixels(cat)); Besides, I run the server using python -m http.server in my command prompt(Windows 10), and this is the error prompt in the console log of my browser: Failed to loa...