Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
L
Lightcurve Database
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 3
    • Issues 3
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Operations
    • Operations
    • Incidents
  • Analytics
    • Analytics
    • Repository
    • Value Stream
  • Members
    • Members
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • William Fong
  • Lightcurve Database
  • Issues
  • #10

Closed
Open
Created Mar 08, 2024 by William Fong@wcfongMaintainer

Bad Multiprocess Pickle of DB Object

Operator ran into this error. Database sessions have registered process guards to prevent access to the same TCP socket. But in this case python is strictly disallowing the object to even begin pickling.

2024.03.08 10:05:51 QLP version 0.12.0b8
2024.03.08 10:06:42 TIC version set to 8.2
2024.03.08 10:07:14 TICA version set to 1.4.0
2024.03.08 10:07:14 Detrending method set to QSP
2024.03.08 10:07:14 KSPBSMAX set to 2.5
2024.03.08 10:07:14 Reading targets for orbit 153
2024.03.08 10:12:42 Found 351810 TIC IDs for cams:ccds 1:1,2,3,4.
2024.03.08 10:12:42 Reading targets for orbit 154
2024.03.08 10:14:40 Found 351797 TIC IDs for cams:ccds 1:1,2,3,4.
2024.03.08 10:14:40 Read 351830 total TICs from orbits (153, 154)
 0%|                                          | 0/351830 [00:00<?, ?it/s]
Traceback (most recent call last):
 File "/pdo/users/petitpas/.local/bin/qlp", line 8, in <module>
  sys.exit(qlp_main())
 File "/pdo/users/petitpas/FFITools/qlp/cli/qlp.py", line 120, in qlp_main
  stage_main(QLPParser)
 File "/pdo/users/petitpas/FFITools/qlp/cli/lctools.py", line 229, in lctools_main
  main(args)
 File "/pdo/users/petitpas/FFITools/qlp/lctools/bin/hlsp.py", line 391, in hlsp_main
  list(tqdm(pool.imap_unordered(h5tofits_wrapper, tasks), total=len(tic_locations)))
 File "/sw/python-versions/python-3.9.2/lib/python3.9/site-packages/tqdm/std.py", line 1178, in __iter__
  for obj in iterable:
 File "/sw/python-versions/python-3.9.2/lib/python3.9/multiprocessing/pool.py", line 870, in next
  raise value
 File "/sw/python-versions/python-3.9.2/lib/python3.9/multiprocessing/pool.py", line 537, in _handle_tasks
  put(task)
 File "/sw/python-versions/python-3.9.2/lib/python3.9/multiprocessing/connection.py", line 211, in send
  self._send_bytes(_ForkingPickler.dumps(obj))
 File "/sw/python-versions/python-3.9.2/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
  cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'create_engine.<locals>.connect'
<petitpas@pdogpu6> (0.11.6) {/scratch/hlsp} %```
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None