Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
61af232
[ADD] queue_job_chunk
kevinkhao May 18, 2020
43f0321
[IMP] Creation logic and saving record state/id/model
kevinkhao Jun 3, 2020
9dc67c5
[IMP] Improved form view
kevinkhao Jun 3, 2020
887663d
[IMP] Logic on queue.job.chunk, company_ids, tests, and component sep…
kevinkhao Jun 4, 2020
1c10d3f
[IMP] Rename reference_company_id to company_id, add company calculat…
kevinkhao Jun 5, 2020
1af01f9
[IMP] Change logic for company_id calculation
kevinkhao Jun 5, 2020
62290fa
Cleanup/README
kevinkhao Jul 4, 2020
2cc2a31
[IMP] run pre-commit
kevinkhao Nov 26, 2020
8987eda
[14.0][MIG] Migration to 14.0
kevinkhao Nov 25, 2020
7dd7a3a
new field stack trace to help failed chunk job
matthieu1611 Apr 15, 2022
c0a3dc6
queue_job_chunk: Migration to 16.0
sebastienbeau Dec 30, 2022
842983c
queue_job_chunk: refactor remove dependency on component
sebastienbeau Dec 30, 2022
146e635
sale_import_base: refactor and migration to V16
sebastienbeau Jan 1, 2023
a25418e
queue_job_chunk: improve code
sebastienbeau Jan 3, 2023
c6ec513
[IMP] queue_job_chunk : Manage retryable Errors
florian-dacosta May 30, 2023
bea479c
queue_job_chunk: fix multicompany rule
hparfr Jul 6, 2023
412a1c6
queue_job_chunk 16.0.0.1.0
github-grap-bot Jul 6, 2023
6d48307
[FIX] run pre-commit for OCA
clementmbr Jan 17, 2025
ab8fdf0
[IMP] backport v16 : pre-commit stuff
clementmbr Jan 20, 2025
ef1a54c
[FIX] backport v16: fix tests for v14
clementmbr Jan 20, 2025
b414bb3
[FIX] run pre-commit and pyproject.toml
nayatec Mar 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions queue_job_chunk/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
===============
Job Queue Chunk
===============

..
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! This file is generated by oca-gen-addon-readme !!
!! changes will be overwritten. !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! source digest: sha256:2ec876f81add7cb79bdb91363bbf23f50aeea267b12cc9e76f6da9cca47d2931
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
:target: https://odoo-community.org/page/development-status
:alt: Beta
.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fqueue-lightgray.png?logo=github
:target: https://github.com/OCA/queue/tree/14.0/queue_job_chunk
:alt: OCA/queue
.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
:target: https://translation.odoo-community.org/projects/queue-14-0/queue-14-0-queue_job_chunk
:alt: Translate me on Weblate
.. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png
:target: https://runboat.odoo-community.org/builds?repo=OCA/queue&target_branch=14.0
:alt: Try me on Runboat

|badge1| |badge2| |badge3| |badge4| |badge5|

Adds the notion of queue job chunks, essentially a queue job with some metadata.

**Table of contents**

.. contents::
:local:

Usage
=====

1. Create a queue job chunk using relevant fields to specify which service to use
2. Use menu to check its status and if you want to, modify its contents to re-run it

Bug Tracker
===========

Bugs are tracked on `GitHub Issues <https://github.com/OCA/queue/issues>`_.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us to smash it by providing a detailed and welcomed
`feedback <https://github.com/OCA/queue/issues/new?body=module:%20queue_job_chunk%0Aversion:%2014.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.

Do not contact contributors directly about support or help with technical issues.

Credits
=======

Authors
~~~~~~~

* Akretion

Contributors
~~~~~~~~~~~~

* Kevin Khao <kevin.khao@akretion.com>
* Sébastien Beau <sebastien.beau@akretion.com>

Other credits
~~~~~~~~~~~~~

The development of this module has been financially supported by:

* Akretion <www.akretion.com>

Maintainers
~~~~~~~~~~~

This module is maintained by the OCA.

.. image:: https://odoo-community.org/logo.png
:alt: Odoo Community Association
:target: https://odoo-community.org

OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.

This module is part of the `OCA/queue <https://github.com/OCA/queue/tree/14.0/queue_job_chunk>`_ project on GitHub.

You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.
1 change: 1 addition & 0 deletions queue_job_chunk/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from . import models
18 changes: 18 additions & 0 deletions queue_job_chunk/__manifest__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Copyright (c) Akretion 2020
# License AGPL-3.0 or later (http://www.gnu.org/licenses/lgpl.html)

{
"name": "Job Queue Chunk",
"version": "14.0.2.0.0",
"author": "Akretion, Odoo Community Association (OCA)",
"website": "https://github.com/OCA/queue",
"license": "AGPL-3",
"category": "Generic Modules",
"depends": ["queue_job"],
"data": [
"views/queue_job_chunk.xml",
"security/security.xml",
"security/ir.model.access.csv",
],
"installable": True,
}
1 change: 1 addition & 0 deletions queue_job_chunk/models/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from . import queue_job_chunk
115 changes: 115 additions & 0 deletions queue_job_chunk/models/queue_job_chunk.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# Copyright (c) Akretion 2020
# License AGPL-3.0 or later (http://www.gnu.org/licenses/lgpl.html)

import json
import traceback

from psycopg2 import OperationalError

from odoo import api, fields, models
from odoo.service.model import PG_CONCURRENCY_ERRORS_TO_RETRY

from odoo.addons.queue_job.exception import RetryableJobError

# Use to bypass chunks entirely for easier debugging
DEBUG_MODE = False


class QueueJobChunk(models.Model):
_name = "queue.job.chunk"
_description = "Queue Job Chunk"
_order = "id desc"

processor = fields.Selection([])
data_str = fields.Text(string="Editable data")
state = fields.Selection(
[("pending", "Pending"), ("done", "Done"), ("fail", "Failed")],
default="pending",
readonly=True,
)
state_info = fields.Text("Additional state information", readonly=True)
model_name = fields.Char(readonly=True)
record_id = fields.Integer(readonly=True)
reference = fields.Reference(
selection="_selection_target_model",
compute="_compute_reference",
store=True,
readonly=True,
)
company_id = fields.Many2one(
"res.company",
compute="_compute_reference",
store=True,
readonly=True,
)
stack_trace = fields.Text(readonly=True)

@api.model
def _selection_target_model(self):
models = self.env["ir.model"].search([])
return [(model.model, model.name) for model in models]

@api.depends("model_name", "record_id")
def _compute_reference(self):
for rec in self:
rec.company_id = self.env.user.company_id
if rec.model_name and rec.record_id:
rec.reference = "{},{}".format(rec.model_name, rec.record_id or 0)
record = self.env[rec.model_name].browse(rec.record_id)
if "company_id" in record._fields:
rec.company_id = record.company_id
else:
rec.reference = False

@api.model_create_multi
def create(self, vals):
result = super().create(vals)
for rec in result:
rec.enqueue_job()
return result

def button_retry(self):
self.enqueue_job()

def enqueue_job(self):
if DEBUG_MODE:
return self.process_chunk()
else:
return self.with_delay().process_chunk()

def _get_processor(self):
# return here whatever class you want
# it can be a pure python class, an odoo TransientModel ...
raise NotImplementedError

def _get_data(self):
return json.loads(self.data_str)

def process_chunk(self):
self.ensure_one()
try:
with self.env.cr.savepoint():
processor = self._get_processor()
result = processor.run()
except RetryableJobError:
raise
except Exception as e:
if DEBUG_MODE:
raise
# TODO maybe it will be simplier to have a kind of inherits
#  on queue.job to avoid a double error management
# so a failling chunk will have a failling job
if (
isinstance(e, OperationalError)
and e.pgcode in PG_CONCURRENCY_ERRORS_TO_RETRY
):
# In that case we raise an error so queue_job
# will do a RetryableJobError
raise
self.state = "fail"
self.state_info = type(e).__name__ + str(e.args)
self.stack_trace = traceback.format_exc()
return False
self.state_info = ""
self.state = "done"
return result
3 changes: 3 additions & 0 deletions queue_job_chunk/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[build-system]
requires = ["whool"]
build-backend = "whool.buildapi"
2 changes: 2 additions & 0 deletions queue_job_chunk/readme/CONTRIBUTORS.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* Kevin Khao <kevin.khao@akretion.com>
* Sébastien Beau <sebastien.beau@akretion.com>
3 changes: 3 additions & 0 deletions queue_job_chunk/readme/CREDITS.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
The development of this module has been financially supported by:

* Akretion <www.akretion.com>
1 change: 1 addition & 0 deletions queue_job_chunk/readme/DESCRIPTION.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Adds the notion of queue job chunks, essentially a queue job with some metadata.
2 changes: 2 additions & 0 deletions queue_job_chunk/readme/USAGE.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
1. Create a queue job chunk using relevant fields to specify which service to use
2. Use menu to check its status and if you want to, modify its contents to re-run it
3 changes: 3 additions & 0 deletions queue_job_chunk/security/ir.model.access.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink
access_queue_job_chunk_manager,queue job manager,model_queue_job_chunk,queue_job.group_queue_job_manager,1,1,1,1
access_queue_job_queue_job_chunk_user,queue job manager,queue_job.model_queue_job,group_queue_job_chunk_user,1,0,0,0
24 changes: 24 additions & 0 deletions queue_job_chunk/security/security.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<?xml version="1.0" encoding="utf-8" ?>
<odoo>

<record id="group_queue_job_chunk_user" model="res.groups">
<field name="name">Job Queue chunk User</field>
<field name="category_id" ref="queue_job.module_category_queue_job" />
</record>

<record id="queue_job.group_queue_job_manager" model="res.groups">
<field
name="implied_ids"
eval="[(4, ref('queue_job_chunk.group_queue_job_chunk_user'))]"
/>
</record>

<record id="queue_job_chunk_comp_rule" model="ir.rule">
<field name="name">Job Queue chunk multi-company</field>
<field name="model_id" ref="model_queue_job_chunk" />
<field
name="domain_force"
>['|', ('company_id','=',False), ('company_id','in',company_ids)]</field>
</record>

</odoo>
Loading