Commit Graph

1508 Commits (master)
 

Author SHA1 Message Date
Thomas Lamprecht 17c45baaa4 bump version to 8.1.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-21 13:04:26 +01:00
Thomas Lamprecht 7ccdc805e3 section config: avoid unamed boolean parameter use hash
Even with just one param it's extra work to check what it refers too,
with named ones in a hash one hasn't that issue even with many params.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-17 09:58:01 +01:00
Dominik Csapak 7d6cc4f14f section config: add tests for separated property lists
more or less a copy from the normal section config test, but now with
properties defined multiple times as well as conflicting options

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ TL: improve consistency with property-isolation terminology ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-17 09:57:39 +01:00
Dominik Csapak 7887b1cb0e section config: allow full property-isolation for plugins
When using 'init(property_isolation => 1)', the code saves the
property lists per type instead of a big one, and using
create/updateSchema creates a new schema with the options as 'oneOf'
and/or 'instance-types' (depending if the schemas match).

With that, we change how we work with the options hash:

It's not needed anymore to specify options that are specified in the
type specific propertyList, except if it's 'fixed => 1' (since that does
not exist in the schema)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ TL: ensure consistency with new property-isolation terminology ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-17 09:57:34 +01:00
Dominik Csapak 942583468f json schema: implement 'oneOf' schema
a schema can now have the 'oneOf' property which is an array of regular
schemas. In the default case any of that has to match. If the
'type-property'/'instance-types' are given, only the schema for the specific
type will be checked (and handles as 'additionalProperties' if there is
no matching type)

the field found in 'type-property' has to be on the same level
(so for oneOf the nested schemas should not include that).

Documentation is adapted so that options are grouped per `type-property=value`
after the regular options (with their individual descriptions/types/etc.)

oneOfs without 'type-property'/'instance-tyeps' simply show up twice for
now with an 'or' line in between.

command line parsing is a bit weird for now since Getopt::Long
can't have multiple variants for the same property (but works fine with
pvesh for our current use cases). it gets shown as '--foo <multiple' if
they are not optional.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-17 09:24:09 +01:00
Dominik Csapak 15645af168 tools: add is_deeply
to compare nested hashes/lists and scalar values recursively.
Also includes some tests

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-17 09:24:09 +01:00
Dominik Csapak e42f1dd6a1 section config: add test for the schemas
by simply doing an 'is_deeply' on the generated schema with
the current generated schema

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-17 09:24:09 +01:00
Filip Schauer fe468fad74 tools: Add mount flag constants
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
2023-11-13 15:08:58 +01:00
Filip Schauer b792e8df81 tools: Add mknod syscall
Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
2023-11-13 15:08:58 +01:00
Thomas Lamprecht 118ca13a88 bump version to 8.0.10
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-07 08:58:34 +01:00
Gabriel Goller a992ba134c fix #4162: added `Auto-Submitted` header to email body
`Auto-Submitted` is defined in the rfc 5436 [1] and describes how
an automatic response (f.e. ooo replies, etc.) should behave on the
emails. When using `Auto-Submitted: auto-generated` (or any value
other than `none`) automatic replies won't be triggered.

[1]: https://www.rfc-editor.org/rfc/rfc3834.html

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2023-11-06 18:39:48 +01:00
Dominik Csapak b4b17fd95b pbs client: add 'tar' parameter to file_restore_extract
so that we can get a 'tar.zst' from proxmox-file-restore by passing
'--format tar --zstd' to the file-restore binary

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-06 18:33:38 +01:00
Thomas Lamprecht 6ebc843904 readme: avoid using deprecated apt-key
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-09-13 13:43:20 +02:00
Stefan Lendl 1edcaa6c1c readme: update mentions of bullseye to bookworm
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
2023-09-13 13:38:04 +02:00
Fiona Ebner b2c87c8857 bump version to 8.0.9
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-09-11 14:12:03 +02:00
Lukas Wagner 2943910ec3 tools: allow to force UTF-8 encoding for file_set_contents
Rationale: This is used from cfs_write_file, which is now also used to
write utf8-encoded strings that come from Rust. If no encoding is
specified while writing the file, we run into problems with certain
special characters (e.g. 'ü').

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
[FG: use flag parameter instead of encoding as a string
     use stricter 'UTF-8' instaed of 'utf8' (see 'perldoc Enocode')]
Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
[FE: implement changes suggested by Fabian
     move binmode call to where $fh is known to be set]
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-09-11 13:42:07 +02:00
Fiona Ebner 41ed439635 run fork with timeout: only special case timeout error in list context
run_with_timeout() will treat a timeout error differently when called
in list context and run_fork_with_timeout() should do the same. Ensure
this by calling run_with_timeout() in list context if and only if
run_fork_with_timeout() is called in list context too.

Fixes: a6aa0ae ("run with timeout: return if timeout happened in list context")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-08-30 16:44:40 +02:00
Fiona Ebner eac8b4b872 run with timeout: only special-case timeout error in list-context
and not other errors too.

Fixes: a6aa0ae ("run with timeout: return if timeout happened in list context")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-08-30 16:44:40 +02:00
Fiona Ebner 9c357928ef test: section config: also test array entries in unknown section
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-08-16 11:25:13 +02:00
Dominik Csapak d2a6411cab SectionConfig: fix handling unknown sections
if we're parsing an unknown section, we cannot check the schema with
`is_array` to check if it's an array type or not, thus we have to
handle that separately.

fix this by handling data in unknown sections like an array similar to
"cb2646c7b4974e33f4148752deec71f0d589b0f3" in proxmox-section-config.
This way we can write unknown section out again like we parsed it.

Add a regression test for an unknown field not in the schema.

This fixes an issue, where calling `qm destroy ID --purge` removed much
of the configs ob backup jobs (since there we parse an 'unknown' section
and run into the `is_array` error)
(Reported in the forum: https://forum.proxmox.com/threads/132091)

Suggested-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-08-16 11:19:26 +02:00
Wolfgang Bumiller 5738817d17 bump version to 8.0.8
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-11 13:25:14 +02:00
Christoph Heiss 741bf653ae section config: allow base properties for {create, update}Schema()
This works the same way as e.g. get_standard_option does it.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2023-08-11 13:22:16 +02:00
Christoph Heiss 10de5bfc2a ldap: handle errors explicitly everywhere instead of simply `die`ing
Most codepaths already have explicit error handling (by the means of
checking the return value), which is essential dead code due to setting
`onerror`.

As LDAP errors might get presented to users due to upcoming changes, the
error location should not be present in these error messages, thus
switch to explicit handling.

Only two calls were missing such explicit handling of errors, so these
are amended as appropriate. Further, some `die`s were missing newlines
at the end of the message, which - again - would cause the error
location to be included.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2023-08-11 13:21:13 +02:00
Philipp Hufnagl 4bb9bfe70b fix whitespaces
Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>

FG: removed hunks that changed alignment..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-04 13:47:10 +02:00
Fabian Grünbichler 0bf2e89a39 download file from url: improve cleanup
don't attempt cleanup if temp files don't exist (anymore, or not yet).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-04 13:47:10 +02:00
Fabian Grünbichler a4df83987b download file from url: simplify error handling
the top-level error handling ensures the temporary downloaded file gets
removed in case of an error, so there is no need to also handle that when
decompression fails..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-04 12:50:40 +02:00
Fabian Grünbichler bf8f0ca200 download file from url: fix indentation
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-04 12:48:23 +02:00
Philipp Hufnagl 7c3e155b28 fix #4849: download file from url: add opt parameter for a decompression command
Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
2023-08-04 12:45:35 +02:00
Wolfgang Bumiller a73f6a80ba bump version to 8.0.7
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-07-24 11:56:11 +02:00
Lukas Wagner 13ae568f8e JSONSchema: increase maxLength of config-digest to 64
The new notification backend is implemented in Rust where we use SHA256
for config digests.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-07-24 11:44:22 +02:00
Thomas Lamprecht f080ddbdc8 bump version to 8.0.6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-07-01 19:24:12 +02:00
Thomas Lamprecht a6aa0ae945 run with timeout: return if timeout happened in list context
This can be relevant info do differentiate if an undef return value
happened due to the closure returning it or if it happened due to a
timeout.

While for quite a few cases this could be handled by a
variable captured by the passed closure code reference, acting as
messenger, that might often require needless wrapping.

Also run_fork_with_timeout warned errors of execution, but any such
error handling for an actual timeout is better handled at the call
site, as a context-less "got timeout" at STDERR or journal is really
not helpful.

I checked all call sites of both, run_fork_with_timeout and
run_with_timeout most do not use the result at all, and the ones that
do are in scalar context.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-07-01 18:45:11 +02:00
Thomas Lamprecht 0c4641dcf8 network: cope with non-existing interfaces config
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-26 14:15:37 +02:00
Thomas Lamprecht 7722f312fc bump version to 8.0.5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-17 13:58:27 +02:00
Thomas Lamprecht 26db1a619b api dump: ignore proxyto_callback code refs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-17 13:52:25 +02:00
Thomas Lamprecht 0827edde8f rest handler: group and sort use statements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-17 13:51:09 +02:00
Thomas Lamprecht a51ba95b78 bump version to 8.0.4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-16 10:29:23 +02:00
Fiona Ebner bca617373a schema: explicitly set min/max for vmid option
The associated pve_verify_vmid() method already restricts the value
to this range, but this wouldn't be visible in the API viewer for
example [0].

The verify method is also called by qemu-server's qmextract, so it's
not possible to just drop the method right now.

[0]: https://forum.proxmox.com/threads/128845/post-564526

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-16 10:21:47 +02:00
Dominik Csapak bc24aad316 fix #4778: fix boolean type check for json parameters over the api
if a real json boolean is sent via the api, $value is a
JSON::PP::Boolean here instead of a string/scalar

so we should validate that too

the $value itself can be used normally in conditions like
----
if ($value) {
----

if the value is stringified, the result is either '1' or '0' so even if
we have some explicit checks like that it should not make a problem

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-06-15 14:58:56 +02:00
Fiona Ebner d4218592e5 syslog: interpret priority level 'warn' correctly
There are some use sites, e.g. HA manager, pvescheduler that
incorrectly use priority level 'warn'. Likely because that is allowed
for some other log helpers in the codebase. Instead of fixing them all
one-by-one, simply allow 'warn' as a priority too.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-14 17:43:13 +02:00
Friedrich Weber 2036990287 ldap: fail authentication if dn is empty
This fixes an issue with LDAP servers that accept anonymous binds with
a non-empty password: If a user exists in the PVE LDAP realm, but PVE
cannot find the corresponding LDAP entry during login, they could log
in with any non-empty password.

This issue affects only LDAP realms. AD realms are not affected
because they perform no username->dn mapping.

At least the following LDAP server configurations seem to accept a
bind with empty DN and non-empty password and are affected:

* OpenLDAP with anonymous binds and the non-default setting
  `olcAllows: bind_anon_cred` enabled.
* AD (when used in an LDAP realm instead of an AD realm). However, for
  the issue to trigger, the LDAP search for the username->dn mapping
  has to succeed but return zero results. This can happen, for
  example, if the LDAP realm has (1) a bind DN set or (2) no bind DN
  set and AD was manually configured to allow anonymous LDAP searches
  for user entries.

The situation that a user exists in the PVE realm but is missing in
the LDAP directory can occur, for example, (1) if the user was created
manually or (2) if the LDAP entry is deleted or the base DN is
changed, but the LDAP realm has not been re-synced with
remove-vanished.

The username->dn mapping is performed by `get_user_dn`, which performs
an LDAP search. If the LDAP search for the user entry succeeds but
returns zero results (e.g. if the entry does not exist), `get_user_dn`
returns undef. Then, `auth_user_dn` is called with $dn being undef and
the user-provided $pw and performs an LDAP simple bind with these
credentials. If $pw is empty, Net::LDAP throws an error, but if it is
non-empty, it performs an LDAP bind with an empty DN and the password
provided by the user. If the LDAP server accepts this bind, the user
is logged in.

To fix this, `auth_user_dn` now dies/returns (depending on the $noerr
parameter) if the dn is falsy, which is the case for undef and the
empty string.

The issue was originally reported by forum user ITKR [0].

[0] https://forum.proxmox.com/threads/128788/

Suggested-by: Dominik Csapak <d.csapak@proxmox.com>
Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Friedrich Weber <f.weber@proxmox.com>
2023-06-13 17:17:31 +02:00
Thomas Lamprecht 1ac0a30a0a read firstline: only map ENOENT to undef, raise error otherwise
Errors like permission denied or I/O ones should bubble up, otherwise
it might hide serious issues and seemingly continue to work, with a
wrong state or the like.

One could argue that the case for not existent should return undef,
while an empty file should return an empty string, but for that we
might want to check all use-sites first.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-06-13 07:16:40 +02:00
Wolfgang Bumiller aa4898ebbe bump version to 8.0.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-07 13:53:25 +02:00
Wolfgang Bumiller 744a1e631b d/control: break qemu-server <<8.0.1 and geust-common <<5.0.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-07 13:53:20 +02:00
Dominik Csapak a26f385865 JSONSchema: disable '-alist' format
this should not be needed anymore since we can now use a simple array
in the api instead

Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-06-07 13:50:53 +02:00
Dominik Csapak 69d9edcc75 section config: implement array support
enables section configs in the style of:

----
type: id
    property value
    property value2
    property value3
----

can be combined with property strings

the provided create and update schema just pass through the array type
to the api, so the api call must always contain the complete array

also adds a test case for such array fields

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-06-07 13:50:51 +02:00
Wolfgang Bumiller 9a3af02069 bump version to 8.0.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-07 13:12:27 +02:00
Dominik Csapak 07f136d636 JSONSchema: add support for array parameter in api calls, cli and config
a few things were missing for it to work:
* on the cli, we have to get the option as an array if the type is an
  array
* the untainting must be done recursively, otherwise, the regex matching
  converts an array hash into the string 'ARRAY(0x123412341234)'
* JSONSchema::parse_config did not handle array formats specially, but
  we want to allow to specify them multiple time
* the biggest point: in the RESTHandler, to be compatible with the
  current gui behavior, we have to rewrite two parameter types:
  - when the api defines a '-list' format for a string type, but we get
    a list (because of the changes in http-server), we join the list
    with a comma into a string
  - when the api defines an 'array' type, but we get a scalar value,
    wrap the value in an array (because for www-form-urlencoded, you
    cannot send an array with a single value) add tests for this
    behavior, some of which we want to deprecate and remove in the
    future

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2023-06-07 13:10:50 +02:00
Stefan Hanreich 7bbfaff1e0 json schema: improve description of bwlimit parameter
This makes the description consistent with the other places that
have bwlimit as a parameter as well.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2023-06-06 17:55:55 +02:00
Wolfgang Bumiller b2bf317fcb remove unused SysFSTools::pci_cleanup_mdev_device
its use was dropped with qemu-server 7.2-7 which we already
have a 'Breaks' entry for, so no bump required there

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-06-05 11:56:29 +02:00