Как удалить даг файл

Начиная с марта 2016 года, многие желающие помайнить столкнулись с ошибкой запуска «CUDA error — cannot allocate big buffer for DAG», которая сообщает, что не удается выделить большой буфер для ДАГ файла. Причина возникшей ситуации — особый алгоритм майнинга эфира и подобных ему криптовалют. Согласно новым принципам работы, для функционирования систем необходимо иметь видеокарту (AMD, NVIDIA), с объемом памяти свыше 2 Гб. В настоящий момент, это минимальный предел.

Ошибка-CUDA-error-11-cannot-allocate-big-buffer-for-DAG

Что такое DAG-файл и почему он растет?

Примечательно, что размер DAG постепенно увеличивается. Согласно используемым алгоритмам, можно определить размер файла в зависимости от текущей эпохи. Под понятием «эпоха» подразумевается цикл прохождения 30 тыс. блоков. Иными словами, у каждого отдельно взятого блока свой размер DAG, а определенному объему этого файла соответствует своя эпоха.

Постепенный рост размера DAG файла способствует тому, что его объем просто не может полностью загрузиться в память видяхи. Эта проблема актуальна для всех криптовалют, использующих алгоритм добычи Dagger Hashimoto. Обратите внимание, что различные валюты находятся на разных эпохах, а скорость прохождения одного цикла для каждой системы своя.

Итак, ошибка указывает на то, что выделить больше памяти нельзя. Стоит заметить, что после загрузки выводится еще одно уведомление «CUDA error 11 — cannot write buffer for DAG», указывающее на то, что запись буфера для DAG остановлена. Способов исправить ошибку, на самом деле, достаточно много. Давайте рассмотрим наиболее популярные варианты.

Выберите-наилучшее-быстродействие

Задайте-значения-1367-16400-Мб

Для видеокарт свыше 2 Гб

NetFramework;
CUDA;
OpenCL;
все необходимые компоненты C++.

Программа-для-удаления-драйверов-Display-Driver-Uninstaller

Прописать в консоли до запуска майнера такие команды:

Если вышеуказанные процедуры не помогли, то нужно искать решение, подходящее именно для вашей модели GPU.

Sorry, you have been blocked

This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

What can I do to resolve this?

You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

Cloudflare Ray ID: 8194e5381f8db377 • Your IP: Click to reveal 45.84.122.38 • Performance & security by Cloudflare

Airflow: how to delete a DAG?

I have started the Airflow webserver and scheduled some dags. I can see the dags on web GUI.

How can I delete a particular DAG from being run and shown in web GUI? Is there an Airflow CLI command to do that?

I looked around but could not find an answer for a simple way of deleting a DAG once it has been loaded and scheduled.

21 Answers 21

Edit 8/27/18 — Airflow 1.10 is now released on PyPI!

How to delete a DAG completely

We have this feature now in Airflow ≥ 1.10!

The PR #2199 (Jira: AIRFLOW-1002) adding DAG removal to Airflow has now been merged which allows fully deleting a DAG’s entries from all of the related tables.

The core delete_dag(. ) code is now part of the experimental API, and there are entrypoints available via the CLI and also via the REST API.

REST API (running webserver locally):

Warning regarding the REST API: Ensure that your Airflow cluster uses authentication in production.

Installing / upgrading to Airflow 1.10 (current)

To upgrade, run either:

Remember to check UPDATING.md first for the full details!

This is my adapted code using PostgresHook with the default connection_id.

DAG-s can be deleted in Airflow 1.10 but the process and sequence of actions must be right. There’s an «egg and chicken problem» — if you delete DAG from frontend while the file is still there the DAG is reloaded (because the file is not deleted). If you delete the file first and refresh the page then DAG cannot be deleted from web gui any more. So the sequence of actions that let me delete a DAG from frontend was:

delete the DAG file (in my case delete from pipeline repository and deploy to airflow servers, esp the scheduler)
DO NOT refresh web GUI.
In the web GUI in the DAGs view (normal frontpage) click on «Delete dag» -> the red icon on the far right.
It cleans up all the remains of this DAG from the database.

Not sure why Apache Airflow doesn’t have an obvious and easy way to delete a DAG

I just wrote a script that deletes everything related to a particular dag, but this is only for MySQL. You can write a different connector method if you are using PostgreSQL. Originally the commands where posted by Lance on https://groups.google.com/forum/#!topic/airbnb_airflow/GVsNsUxPRC0 I just put it in script. Hope this helps. Format: python script.py dag_id

Airflow 1.10.1 has been released. This release adds the ability to delete a DAG from the web UI after you have deleted the corresponding DAG from the file system.

See this ticket for more details:

[AIRFLOW-2657] Add ability to delete DAG from web ui

Airflow Links menu with delete icon

Please note that this doesn’t actually delete the DAG from the file system, you will need to do this manually first otherwise the DAG will get reloaded.

I’ve written a script that deletes all metadata related to a specific dag for the default SQLite DB. This is based on Jesus’s answer above but adapted from Postgres to SQLite. Users should set ../airflow.db to wherever script.py is stored relative to the default airflow.db file (usually

/airflow ). To execute, use python script.py dag_id .

For those who have direct access to the Postgres psql console of the airflow db, you can simply execute the following request to remove the DAG:

A similar (with minor changes) query is suitable for other databases, such as MySQL and SQLite.

There is nothing inbuilt in Airflow that does that for you. In order to delete the DAG, delete it from the repository and delete the database entries in the Airflow metastore table — dag.

Based on the answer of @OlegYamin, I’m doing the following to delete a dag backed by postgres, where airflow uses the public schema.

WARNING: The effect/correctness of the first delete query is unknown to me. It is just an assumption that it is needed.

just delete it from mysql, works fine for me. delete them from below tables:

(might be more tables in future release) then restart webserver and worker.

I have airflow version 1.10.2 and I tried executing airflow delete_dag command but the command throws following error:

bash-4.2# airflow delete_dag dag_id

[2019-03-16 15:37:20,804] INFO — settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800, pid=28224 /usr/lib64/python2.7/site-packages/psycopg2/init.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use «pip install psycopg2-binary» instead. For details see: http://initd.org/psycopg/docs/install.html#binary-install-from-pypi. «»») This will drop all existing records related to the specified DAG. Proceed? (y/n)y Traceback (most recent call last): File «/usr/bin/airflow», line 32, in args.func(args) File «/usr/lib/python2.7/site-packages/airflow/utils/cli.py», line 74, in wrapper return f(*args, **kwargs) File «/usr/lib/python2.7/site-packages/airflow/bin/cli.py», line 258, in delete_dag raise AirflowException(err) airflow.exceptions.AirflowException: Server error

Though I am able to delete through Curl command. Please let me know if anyone have idea about this command’s execution, is this known or I am doing something wrong.

There is not a command to delete a dag, so you need to first delete the dag file, and then delete all the references to the dag_id from the airflow metadata database.

WARNING

You can reset the airflow meta database, you will erase everything, including the dags, but remember that you will also erase the history, pools, variables, etc.